UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection

Petke, J; Cohen, MB; Harman, M; Yoo, S; (2015) Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection. IEEE Transactions on Software Engineering , 41 (9) pp. 901-924. 10.1109/TSE.2015.2421279. Green open access

[thumbnail of 07081752.pdf]
Preview
Text
07081752.pdf

Download (3MB) | Preview

Abstract

Combinatorial interaction testing (CIT) is important because it tests the interactions between the many features and parameters that make up the configuration space of software systems. Simulated Annealing (SA) and Greedy Algorithms have been widely used to find CIT test suites. From the literature, there is a widely-held belief that SA is slower, but produces more effective tests suites than Greedy and that SA cannot scale to higher strength coverage. We evaluated both algorithms on seven real-world subjects for the well-studied two-way up to the rarely-studied six-way interaction strengths. Our findings present evidence to challenge this current orthodoxy: real-world constraints allow SA to achieve higher strengths. Furthermore, there was no evidence that Greedy was less effective (in terms of time to fault revelation) compared to SA; the results for the greedy algorithm are actually slightly superior. However, the results are critically dependent on the approach adopted to constraint handling. Moreover, we have also evaluated a genetic algorithm for constrained CIT test suite generation. This is the first time strengths higher than 3 and constraint handling have been used to evaluate GA. Our results show that GA is competitive only for pairwise testing for subjects with a small number of constraints.

Type: Article
Title: Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/TSE.2015.2421279
Publisher version: http://dx.doi.org/10.1109/TSE.2015.2421279
Language: English
Additional information: This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/licenses/by/3.0/
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1471645
Downloads since deposit
188Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item