Joffe, L;
Clark, D;
(2019)
Directing a Search Towards Execution Properties with a Learned Fitness Function.
In:
Proceedings of IEEE International Conference on Software Testing, Verification and Validation - 2019.
(pp. pp. 206-216).
IEEE
Preview |
Text
main.pdf - Accepted Version Download (905kB) | Preview |
Abstract
Search based software testing is a popular and successful approach both in academia and industry. SBST methods typically aim to increase coverage whereas searching for executions with specific properties is largely unresearched. Fitness functions for execution properties often possess search landscapes that are difficult or intractable. We demonstrate how machine learning techniques can convert a property that is not searchable, in this case crashes, into one that is. Through experimentation on 6000 C programs drawn from the Codeflaws repository, we demonstrate a strong, program independent correlation between crashing executions and library function call patterns within those executions as discovered by a neural net. We then exploit the correlation to produce a searchable fitness landscape to modify American Fuzzy Lop, a widely used fuzz testing tool. On a test set of previously unseen programs drawn from Codeflaws, a search strategy based on a crash targeting fitness function outperformed a baseline in 80.1% of cases. The experiments were then repeated on three real world programs: the VLC media player, and the libjpeg and mpg321 libraries. The correlation between library call traces and crashes generalises as indicated by ROC AUC scores of 0.91, 0.88 and 0.61. The produced search landscape however is not convenient due to plateaus. This is likely because these programs do not use standard C libraries as often as do those in Codeflaws. This limitation can be overcome by considering a more powerful observation domain and a broader training corpus in future work. Despite limited generalisability of the experimental setup, this research opens new possibilities in the intersection of machine learning, fitness functions, and search based testing in general.
| Type: | Proceedings paper |
|---|---|
| Title: | Directing a Search Towards Execution Properties with a Learned Fitness Function |
| Event: | IEEE International Conference on Software Testing, Verification and Validation - 2019 |
| Location: | Xi'an, China |
| Dates: | 22 April 2019 - 27 April 2019 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/ICST.2019.00029 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Search based software engineering, Search based software testing, Fuzzing, Machine learning |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10069676 |
Archive Staff Only
 |
View Item |
