UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Comparing white-box and black-box test prioritization

Henard, C; Papadakis, M; Harman, M; Jia, Y; Traon, YL; (2016) Comparing white-box and black-box test prioritization. In: Dillon, L and Visser, W and Williams, L, (eds.) ICSE '16: Proceedings of the 38th International Conference on Software Engineering. (pp. pp. 523-534). Association for Computing Machinery (ACM): New York. Green open access

[thumbnail of ICSE16A.pdf]
Preview
Text
ICSE16A.pdf - Published Version

Download (814kB) | Preview

Abstract

Although white-box regression test prioritization has been well-studied, the more recently introduced black-box prioritization approaches have neither been compared against each other nor against more well-established white-box techniques. We present a comprehensive experimental comparison of several test prioritization techniques, including wellestablished white-box strategies and more recently introduced black-box approaches. We found that Combinatorial Interaction Testing and diversity-based techniques (Input Model Diversity and Input Test Set Diameter) perform best among the black-box approaches. Perhaps surprisingly, we found little difference between black-box and white-box performance (at most 4% fault detection rate difference). We also found the overlap between black-and white-box faults to be high: the first 10% of the prioritized test suites already agree on at least 60% of the faults found. These are positive findings for practicing regression testers who may not have source code available, thereby making white-box techniques inapplicable. We also found evidence that both black-box and white-box prioritization remain robust over multiple system releases.

Type: Proceedings paper
Title: Comparing white-box and black-box test prioritization
Event: ICSE '16: 38th International Conference on Software Engineering, 14-22 May 2016, Austin, Texas, USA
ISBN-13: 9781450339001
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/2884781.2884791
Publisher version: http://dx.doi.org/10.1145/2884781.2884791
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Regression Testing, White-box, Black-box
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1502193
Downloads since deposit
1,206Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item