Chekam, TT;
Papadakis, M;
Le Traon, Y;
Harman, M;
(2017)
An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption.
In:
2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).
(pp. pp. 597-608).
IEEE
Preview |
Text
Chekam_ICSE17.pdf - Accepted Version Download (655kB) | Preview |
Abstract
Many studies suggest using coverage concepts, such as branch coverage, as the starting point of testing, while others as the most prominent test quality indicator. Yet the relationship between coverage and fault-revelation remains unknown, yielding uncertainty and controversy. Most previous studies rely on the Clean Program Assumption, that a test suite will obtain similar coverage for both faulty and fixed ('clean') program versions. This assumption may appear intuitive, especially for bugs that denote small semantic deviations. However, we present evidence that the Clean Program Assumption does not always hold, thereby raising a critical threat to the validity of previous results. We then conducted a study using a robust experimental methodology that avoids this threat to validity, from which our primary finding is that strong mutation testing has the highest fault revelation of four widely-used criteria. Our findings also revealed that fault revelation starts to increase significantly only once relatively high levels of coverage are attained.
| Type: | Proceedings paper |
|---|---|
| Title: | An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation that Avoids the Unreliable Clean Program Assumption |
| Event: | 39th IEEE/ACM International Conference on Software Engineering (ICSE), 20-28 May 2017, Buenos Aires, Argentina |
| Location: | Buenos Aires, ARGENTINA |
| Dates: | 20 May 2017 - 28 May 2017 |
| ISBN-13: | 978-1-5386-3868-2 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/ICSE.2017.61 |
| Publisher version: | https://doi.org/10.1109/ICSE.2017.61 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Testing, Java, Correlation, Robustness, Standards, Tools, Mutation testing, test effectiveness, code coverage, real faults, test adequacy |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10058915 |
Archive Staff Only
 |
View Item |
