Zhang, J; Zhang, L; Hao, D; Zhang, L; Harman, M; (2019) An Empirical Comparison of Mutant Selection Assessment Metrics. In: Proceedings of the 14th IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) 2019. IEEE: Xi'an, China, China.

Preview

Text Zhang_An Empirical Comparison of Mutant Selection Assessment Metrics_AAM.pdf - Accepted Version Download (614kB) | Preview

Abstract

Mutation testing is expensive due to the large number of mutants, a problem typically tackled using selective techniques, thereby raising the fundamental question of how to evaluate the selection process. Existing mutant selection approaches rely on one of two types of metrics (or assessment criteria), one based on adequate test sets and the other based on inadequate test sets. This raises the question as to whether these two metrics are correlated, complementary or substitutable for one another. The tester's faith in mutant selection as well as the validity of previous research work using only one metric rely on the answer to this question, yet it currently remains unanswered. To answer it, we perform qualitative and quantitative comparisons with 104 different projects, consisting of over 600,000 lines of code. Our results indicate a strong connection between the two types of metrics (R 2 =0.8622 on average). The strategy for dealing with equivalent mutants and test density is observed to have a negligible impact for mutant selection.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item