TY  - GEN
CY  - New York
N1  - This version is the author accepted manuscript. For information on re-use, please refer to the publisher?s terms and conditions.
ID  - discovery1502193
UR  - http://dx.doi.org/10.1145/2884781.2884791
A1  - Henard, C
A1  - Papadakis, M
A1  - Harman, M
A1  - Jia, Y
A1  - Traon, YL
Y1  - 2016/05//
N2  - Although white-box regression test prioritization has been well-studied, the more recently introduced black-box prioritization approaches have neither been compared against each other nor against more well-established white-box techniques. We present a comprehensive experimental comparison of several test prioritization techniques, including wellestablished white-box strategies and more recently introduced black-box approaches. We found that Combinatorial Interaction Testing and diversity-based techniques (Input Model Diversity and Input Test Set Diameter) perform best among the black-box approaches. Perhaps surprisingly, we found little difference between black-box and white-box performance (at most 4% fault detection rate difference). We also found the overlap between black-and white-box faults to be high: the first 10% of the prioritized test suites already agree on at least 60% of the faults found. These are positive findings for practicing regression testers who may not have source code available, thereby making white-box techniques inapplicable. We also found evidence that both black-box and white-box prioritization remain robust over multiple system releases.
PB  - Association for Computing Machinery (ACM)
T3  - International Conference on Software Engineering
KW  - Regression Testing
KW  -  White-box
KW  -  Black-box
SN  - 0270-5257
TI  - Comparing white-box and black-box test prioritization
SP  - 523
AV  - public
EP  - 534
ER  -