UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Program Transformation Landscapes for Automated Program Modification Using Gin

Petke, Justyna; Alexander, Bradley; Barr, Earl; Brownlee, Alexander EI; Wagner, Markus; White, David R; (2023) Program Transformation Landscapes for Automated Program Modification Using Gin. Empirical Software Engineering , 28 , Article 104. 10.1007/s10664-023-10344-5. Green open access

[thumbnail of ginpaper.pdf]
Preview
Text
ginpaper.pdf - Accepted Version

Download (689kB) | Preview

Abstract

Automated program modification underlies two successful research areas — genetic improvement and program repair. Under the generate-and-validate strategy, automated program modification transforms a program, then validates the result against a test suite. Much work has focused on the search space of application of single fine-grained operators — COPY, DELETE, REPLACE, and SWAP at both line and statement granularity. This work explores the limits of this strategy. We scale up existing findings an order of magnitude from small corpora to 10 real-world Java programs comprising up to 500k LoC. We decisively show that the grammar-specificity of statement granular edits pays off: its pass rate triples that of line edits and uses 10% less computational resources. We confirm previous findings that DELETE is the most effective operator for creating test-suite equivalent program variants. We go farther than prior work by exploring the limits of DELETE ’s effectiveness by exhaustively applying it. We show this strategy is too costly in practice to be used to search for improved software variants. We further find that pass rates drop from 12–34% for single statement edits to 2–6% for 5-edit sequences, which implies that further progress will need human-inspired operators that target specific faults or improvements. A program is amenable to automated modification to the extent to which automatically editing it is likely to produce test-suite passing variants. We are the first to systematically search for a code measure that correlates with a program’s amenability to automated modification. We found no strong correlations, leaving the question open.

Type: Article
Title: Program Transformation Landscapes for Automated Program Modification Using Gin
Open access status: An open access version is available from UCL Discovery
DOI: 10.1007/s10664-023-10344-5
Publisher version: https://doi.org/10.1007/s10664-023-10344-5
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Automated program modification, Genetic improvement, Automated program repair, Search-based software engineering
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10170968
Downloads since deposit
3Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item