Towards Regression Testing Constraints

Regression testing is one of the most crucial and expensive testing process used to validate modified software and detect new faults introduced by into previously modified and tested code. To reduce the cost of regression testing, software testers may choose to prioritize their test cases to run the "important" ones, chosen by some metrics and constraints, earlier in the testing process. Various approaches have been introduced by previous researches in the form of prioritization technique focusing on specific goals of regression testing. One goal of prioritization is to increase a test suite's rate of fault detection. In this context, previous studies have shown that several prioritization techniques can significantly improve rate of fault detection, but these studies have also shown that the effectiveness of these techniques is relative to the assumptions made by the researchers concerning the testing environment. The variations observed in these experiments have mostly been linked to the subject program, test suites characteristics. This makes it difficult for testers to appropriately choose the correct prioritization technique for their testing scenarios. In this paper, regression prioritization techniques are described. Their performance, in fault detection rate, is assessed and the metrics used to assess the effect of variation through discussing experiments done in this context are specified. Then, the results are analyzed and insight about prioritization techniques selection under these constraints is provided.