eprintid: 10184114
rev_number: 7
eprint_status: archive
userid: 699
dir: disk0/10/18/41/14
datestamp: 2024-02-26 17:14:47
lastmod: 2024-02-26 17:14:47
status_changed: 2024-02-26 17:14:47
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: Yi, Jooyong
creators_name: Tan, Shin Hwei
creators_name: Mechtaev, Sergey
creators_name: Bohme, Marcel
creators_name: Roychoudhury, Abhik
title: A correlation study between automated program repair and test-suite metrics
ispublished: pub
divisions: UCL
divisions: B04
divisions: C05
divisions: F48
keywords: Science & Technology, Technology, Computer Science, Software Engineering, Computer Science, Automated program repair, Test suite, Empirical evaluation, Correlation, MUTATION
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: Automated program repair is increasingly gaining traction, due to its potential to reduce debugging cost greatly. The feasibility of automated program repair has been shown in a number of works, and the research focus is gradually shifting toward the quality of generated patches. One promising direction is to control the quality of generated patches by controlling the quality of test-suites used for automated program repair. In this paper, we ask the following research question: “Can traditional test-suite metrics proposed for the purpose of software testing also be used for the purpose of automated program repair?” We empirically investigate whether traditional test-suite metrics such as statement/branch coverage and mutation score are effective in controlling the reliability of generated repairs (the likelihood that repairs cause regression errors). We conduct the largest-scale experiments of this kind to date with real-world software, and for the first time perform a correlation study between various test-suite metrics and the reliability of generated repairs. Our results show that in general, with the increase of traditional test suite metrics, the reliability of repairs tend to increase. In particular, such a trend is most strongly observed in statement coverage. Our results imply that the traditional test suite metrics proposed for software testing can also be used for automated program repair to improve the reliability of repairs.
date: 2018-10-01
date_type: published
publisher: SPRINGER
official_url: https://doi.org/10.1007/s10664-017-9552-y
oa_status: green
full_text_type: other
language: eng
primo: open
primo_central: open_green
verified: verified_manual
elements_id: 1588823
doi: 10.1007/s10664-017-9552-y
lyricists_name: Mechtaev, Sergey
lyricists_id: SMECH63
actors_name: Mechtaev, Sergey
actors_id: SMECH63
actors_role: owner
funding_acknowledgements: NRF2014NCR-NCR001-21 [National Research Foundation, Prime Minister's Office, Singapore under National Cybersecurity R&D Program (TSUNAMi project)]; [Innopolis University]
full_text_status: public
publication: Empirical Software Engineering
volume: 23
number: 5
article_number: 5
pagerange: 2948-2979
pages: 32
issn: 1382-3256
citation:        Yi, Jooyong;    Tan, Shin Hwei;    Mechtaev, Sergey;    Bohme, Marcel;    Roychoudhury, Abhik;      (2018)    A correlation study between automated program repair and test-suite metrics.                   Empirical Software Engineering , 23  (5)    , Article 5.  10.1007/s10664-017-9552-y <https://doi.org/10.1007/s10664-017-9552-y>.       Green open access   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10184114/1/emse17.pdf