UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

An analysis of the relationship between conditional entropy and failed error propagation in software testing

Androutsopoulos, K; Clark, D; Dan, H; Hierons, RM; Harman, M; (2014) An analysis of the relationship between conditional entropy and failed error propagation in software testing. In: Jalote, P and Briand, L and Van der Hoek, A, (eds.) ICSE 2014: Proceedings of the 36th International Conference on Software Engineering. (pp. pp. 573-583). Association for Computing Machinery (ACM): New York, NY, USA. Green open access

[thumbnail of icse_sq_rob.pdf] Text
icse_sq_rob.pdf
Available under License : See the attached licence file.

Download (533kB)

Abstract

Failed error propagation (FEP) is known to hamper software testing, yet it remains poorly understood. We introduce an information theoretic formulation of FEP that is based on measures of conditional entropy. This formulation considers the situation in which we are interested in the potential for an incorrect program state at statement s to fail to propagate to incorrect output. We define five metrics that differ in two ways: whether we only consider parts of the program that can be reached after executing s and whether we restrict attention to a single program path of interest .We give the results of experiments in which it was found that on average one in 10 tests suffered from FEP, earlier studies having shown that this figure can vary significantly between programs. The experiments also showed that our metrics are well-correlated with FEP. Our empirical study involved 30 programs, for which we executed a total of 7,140,000 test cases. The results reveal that the metrics differ in their performance but the Spearman rank correlation with failed error propagation is close to 0.95 for two of the metrics. These strong correlations in an experimental setting, in which all information about both FEP and conditional entropy is known, open up the possibility in the longer term of devising inexpensive information theory based metrics that allow us to minimise the effect of FEP.

Type: Proceedings paper
Title: An analysis of the relationship between conditional entropy and failed error propagation in software testing
Event: 36th International Conference on Software Engineering (ICSE 2014)
ISBN-13: 9781450327565
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/2568225.2568314
Publisher version: http://dx.doi.org/10.1145/2568225.2568314
Language: English
Additional information: Copyright © 2014 ACM
Keywords: Program Analysis, Information Theory
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1429464
Downloads since deposit
288Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item