Brownlee, Alexander EI;
Callan, James;
Even-Mendoza, Karine;
Geiger, Alina;
Hanna, Carol;
Petke, Justyna;
Sarro, Federica;
(2023)
Enhancing Genetic Improvement Mutations Using Large Language Models.
In:
SSBSE 2023: Search-Based Software Engineering. 15th International Symposium.
Springer
Preview |
Text
main.pdf - Accepted Version Download (194kB) | Preview |
Abstract
Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs as mutation operators for GI to improve the search process. We expand the Gin Java GI toolkit to call OpenAI’s API to generate edits for the JCodec tool. We randomly sample the space of edits using 5 different edit types. We find that the number of patches passing unit tests is up to 75% higher with LLM-based edits than with standard Insert edits. Further, we observe that the patches found with LLMs are generally less diverse compared to standard edits. We ran GI with local search to find runtime improvements. Although many improving patches are found by LLM-enhanced GI, the best improving patch was found by standard GI.
| Type: | Proceedings paper |
|---|---|
| Title: | Enhancing Genetic Improvement Mutations Using Large Language Models |
| Event: | SSBSE 2023: Search-Based Software Engineering. 15th International Symposium |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/978-3-031-48796-5_13 |
| Publisher version: | https://doi.org/10.1007/978-3-031-48796-5_13 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. - For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10180563 |
Archive Staff Only
 |
View Item |
