Zhang, J;
Zhang, LZ;
Hao, D;
Wang, M;
Zhang, L;
(2019)
Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness?
In:
Proceedings of the 12th IEEE Conference on Software Testing, Validation and Verification (ICST) 2019.
(pp. pp. 252-263).
IEEE: Xi'an, China, China.
Preview |
Text
Zhang_Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness_AAM.pdf - Accepted Version Download (725kB) | Preview |
Abstract
Code coverage is the most widely adopted criteria for measuring test effectiveness in software quality assurance. The performance of coverage criteria (in indicating test suites’ effectiveness) has been widely studied in prior work. Most of the studies use randomly constructed pseudo test suites to facilitate data collection for correlation analysis, yet no previous work has systematically studied whether pseudo test suites would lead to inflated correlation results. This paper focuses on the potentially wide-spread threat with a study over 123 real-world Java projects. Following the typical experimental process of studying coverage criteria, we investigate the correlation between statement/assertion coverage and mutation score using both pseudo and original test suites. Except for direct correlation analysis, we control the number of assertions and the test suite size to conduct partial correlation analysis. The results reveal that 1) the correlation (between coverage criteria and mutation score) derived from pseudo test suites is much higher than from original test suites (from 0.21 to 0.39 higher in Kendall τb value); 2) contrary to previously reported, statement coverage has a stronger correlation with mutation score than assertion coverage.
| Type: | Proceedings paper |
|---|---|
| Title: | Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness? |
| Event: | 12th IEEE Conference on Software Testing, Validation and Verification (ICST) |
| Location: | Xi'an, China |
| Dates: | 24 April 2019 - 27 April 2019 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/ICST.2019.00033 |
| Publisher version: | http://dx.doi.org/10.1109/ICST.2019.00033 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | test suites, coverage criteria, empirical study |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10075196 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
