UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption

Chekam, TT; Papadakis, M; Le Traon, Y; Harman, M; (2017) An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption. In: 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). (pp. pp. 597-608). IEEE Green open access

[img]
Preview
Text
Chekam_ICSE17.pdf - Accepted version

Download (655kB) | Preview

Abstract

Many studies suggest using coverage concepts, such as branch coverage, as the starting point of testing, while others as the most prominent test quality indicator. Yet the relationship between coverage and fault-revelation remains unknown, yielding uncertainty and controversy. Most previous studies rely on the Clean Program Assumption, that a test suite will obtain similar coverage for both faulty and fixed ('clean') program versions. This assumption may appear intuitive, especially for bugs that denote small semantic deviations. However, we present evidence that the Clean Program Assumption does not always hold, thereby raising a critical threat to the validity of previous results. We then conducted a study using a robust experimental methodology that avoids this threat to validity, from which our primary finding is that strong mutation testing has the highest fault revelation of four widely-used criteria. Our findings also revealed that fault revelation starts to increase significantly only once relatively high levels of coverage are attained.

Type: Proceedings paper
Title: An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption
Event: 39th IEEE/ACM International Conference on Software Engineering (ICSE), 20-28 May 2017, Buenos Aires, Argentina
Location: Buenos Aires, ARGENTINA
Dates: 20 May 2017 - 28 May 2017
ISBN-13: 978-1-5386-3868-2
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ICSE.2017.61
Publisher version: https://doi.org/10.1109/ICSE.2017.61
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Testing, Java, Correlation, Robustness, Standards, Tools, Mutation testing, test effectiveness, code coverage, real faults, test adequacy
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10058915
Downloads since deposit
94Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item