https://discovery.ucl.ac.uk/id/eprint/1502193 2020-02-12T17:21:14Z info:eu-repo/semantics/descriptiveMetadata Comparing white-box and black-box test prioritization Henard, C Papadakis, M Harman, M Jia, Y Traon, YL Although white-box regression test prioritization has been well-studied, the more recently introduced black-box prioritization approaches have neither been compared against each other nor against more well-established white-box techniques. We present a comprehensive experimental comparison of several test prioritization techniques, including wellestablished white-box strategies and more recently introduced black-box approaches. We found that Combinatorial Interaction Testing and diversity-based techniques (Input Model Diversity and Input Test Set Diameter) perform best among the black-box approaches. Perhaps surprisingly, we found little difference between black-box and white-box performance (at most 4% fault detection rate difference). We also found the overlap between black-and white-box faults to be high: the first 10% of the prioritized test suites already agree on at least 60% of the faults found. These are positive findings for practicing regression testers who may not have source code available, thereby making white-box techniques inapplicable. We also found evidence that both black-box and white-box prioritization remain robust over multiple system releases. Regression Testing, White-box, Black-box Association for Computing Machinery (ACM) Dillon, L Visser, W Williams, L 2016-05 Proceedings paper eng In: Dillon, L and Visser, W and Williams, L, (eds.) ICSE '16: Proceedings of the 38th International Conference on Software Engineering. (pp. pp. 523-534). Association for Computing Machinery (ACM): New York. (2016) text https://discovery.ucl.ac.uk/id/eprint/1502193/1/ICSE16A.pdf https://discovery.ucl.ac.uk/id/eprint/1502193/ open info:eu-repo/semantics/objectFile info:eu-repo/semantics/humanStartPage