Ojdanic, M;
Papadakis, M;
Harman, M;
(2023)
Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants.
In:
ESEC/FSE 2023 - Proceedings of the 31st ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
(pp. pp. 2067-2071).
ACM: San Francisco, CA.
Preview |
PDF
3611643.3613089.pdf - Published Version Download (615kB) | Preview |
Abstract
Mutation testing has been demonstrated to be one of the most powerful fault-revealing tools in the tester's tool kit. Much previous work implicitly assumed it to be sufficient to re-compute mutant suites per release. Sadly, this makes mutation results inconsistent; mutant scores from each release cannot be directly compared, making it harder to measure test improvement. Furthermore, regular code change means that a mutant suite's relevance will naturally degrade over time. We measure this degradation in relevance for 143,500 mutants in 4 non-trivial systems, finding that 52% degrade, on average. We introduce a mutant brittleness measure and use it to audit software systems and their mutation suites. We also demonstrate how consistent-by-construction long-standing mutant suites can be identified with a 10x improvement in mutant relevance over an arbitrary test suite. Our results indicate that the research community should avoid the re-computation of mutant suites and focus, instead, on long-standing mutants, thereby improving the consistency and relevance of mutation testing.
| Type: | Proceedings paper |
|---|---|
| Title: | Keeping Mutation Test Suites Consistent and Relevant with Long-Standing Mutants |
| Event: | ESEC/FSE '23: 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1145/3611643.3613089 |
| Publisher version: | http://dx.doi.org/10.1145/3611643.3613089 |
| Language: | English |
| Additional information: | This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third-party material in this article are included in the Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ |
| Keywords: | Evolving Systems, Mutation Testing, Test Adequacy, Continuous Integration, Software Testing |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10184856 |
Archive Staff Only
 |
View Item |
