Jahangirova, G;
Clark, D;
Harman, M;
Tonella, P;
(2019)
An Empirical Validation of Oracle Improvement.
IEEE Transactions on Software Engineering
10.1109/TSE.2019.2934409.
(In press).
Preview |
Text
Clark_An Empirical Validation of Oracle Improvement_AAM.pdf - Accepted Version Download (2MB) | Preview |
Abstract
We propose a human-in-the-loop approach for oracle improvement and analyse whether the proposed oracle improvement process is helping developers to create better oracles. For this, we conducted two human studies with 68 participants overall: an oracle assessment study and an oracle improvement study. Our results show that developers exhibit poor performance (29% accuracy) when manually assessing whether an assertion oracle contains a false positive, a false negative or none of the two. This shows that automated detection of these oracle deficiencies is beneficial for the users. Our tool OASIs (Oracle ASsessment and Improvement) helps developers produce assertions with higher quality. Participants who used OASIs in the improvement study were able to achieve 33% of full and 67% of partial correctness as opposed to participants without the tool who achieved only 21% of full and 43% of partial correctness.
| Type: | Article |
|---|---|
| Title: | An Empirical Validation of Oracle Improvement |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/TSE.2019.2934409 |
| Publisher version: | http://doi.org/10.1109/TSE.2019.2934409 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10079134 |
Archive Staff Only
 |
View Item |
