TY  - INPR
A1  - Jahangirova, G
A1  - Clark, D
A1  - Harman, M
A1  - Tonella, P
SN  - 0098-5589
ID  - discovery10079134
AV  - public
TI  - An Empirical Validation of Oracle Improvement
JF  - IEEE Transactions on Software Engineering
N1  - This version is the author accepted manuscript. For information on re-use, please refer to the publisher?s terms and conditions.
UR  - http://doi.org/10.1109/TSE.2019.2934409
Y1  - 2019/08/31/
N2  - We propose a human-in-the-loop approach for oracle improvement and analyse whether the proposed oracle improvement process is helping developers to create better oracles. For this, we conducted two human studies with 68 participants overall: an oracle assessment study and an oracle improvement study. Our results show that developers exhibit poor performance (29% accuracy) when manually assessing whether an assertion oracle contains a false positive, a false negative or none of the two. This shows that automated detection of these oracle deficiencies is beneficial for the users. Our tool OASIs (Oracle ASsessment and Improvement) helps developers produce assertions with higher quality. Participants who used OASIs in the improvement study were able to achieve 33% of full and 67% of partial correctness as opposed to participants without the tool who achieved only 21% of full and 43% of partial correctness.
PB  - Institute of Electrical and Electronics Engineers
ER  -