Petke, Justyna;
Alexander, Bradley;
Barr, Earl;
Brownlee, Alexander EI;
Wagner, Markus;
White, David R;
(2023)
Program Transformation Landscapes for Automated Program Modification Using Gin.
Empirical Software Engineering
, 28
, Article 104. 10.1007/s10664-023-10344-5.
Preview |
Text
ginpaper.pdf - Accepted Version Download (689kB) | Preview |
Abstract
Automated program modification underlies two successful research areas — genetic improvement and program repair. Under the generate-and-validate strategy, automated program modification transforms a program, then validates the result against a test suite. Much work has focused on the search space of application of single fine-grained operators — COPY, DELETE, REPLACE, and SWAP at both line and statement granularity. This work explores the limits of this strategy. We scale up existing findings an order of magnitude from small corpora to 10 real-world Java programs comprising up to 500k LoC. We decisively show that the grammar-specificity of statement granular edits pays off: its pass rate triples that of line edits and uses 10% less computational resources. We confirm previous findings that DELETE is the most effective operator for creating test-suite equivalent program variants. We go farther than prior work by exploring the limits of DELETE ’s effectiveness by exhaustively applying it. We show this strategy is too costly in practice to be used to search for improved software variants. We further find that pass rates drop from 12–34% for single statement edits to 2–6% for 5-edit sequences, which implies that further progress will need human-inspired operators that target specific faults or improvements. A program is amenable to automated modification to the extent to which automatically editing it is likely to produce test-suite passing variants. We are the first to systematically search for a code measure that correlates with a program’s amenability to automated modification. We found no strong correlations, leaving the question open.
| Type: | Article |
|---|---|
| Title: | Program Transformation Landscapes for Automated Program Modification Using Gin |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/s10664-023-10344-5 |
| Publisher version: | https://doi.org/10.1007/s10664-023-10344-5 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Automated program modification, Genetic improvement, Automated program repair, Search-based software engineering |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10170968 |
Archive Staff Only
 |
View Item |
