Asprone, Dario;
Metzman, Jonathan;
Arya, Abhishek;
Guizzo, Giovani;
Sarro, Federica;
(2022)
Comparing Fuzzers on a Level Playing Field with FuzzBench.
In:
Proceedings of the International Conference on Software Testing, ICST.
IEEE
(In press).
Preview |
Text
Comparing Fuzzers on a Level Playing Field with FuzzBench.pdf - Accepted Version Download (266kB) | Preview |
Abstract
Fuzzing is a testing approach commonly used in industry to discover bugs in a given software under test (SUT). It consists of running a SUT iteratively with randomly generated (or mutated) inputs, in order to find as many as possible inputs that make the SUT crash. Many fuzzers have been proposed to date, however no consensus has been reached on how to properly evaluate and compare fuzzers. In this work we evaluate and compare nine prominent fuzzers by carrying out a thorough empirical study based on an open-source framework developed by Google, namely FuzzBench, and a manually curated benchmark suite of 12 real-world software systems. The results show that honggfuzz and AFL++ are, in that order, the best choices in terms of general purpose fuzzing effectiveness. The results also show that none of the fuzzers outperforms the others in terms of efficiency across all considered metrics, that no particular bug affinity is found for any fuzzer, and that the correlation found between coverage and number of bugs depends more on the SUT rather than on the fuzzer used.
| Type: | Proceedings paper |
|---|---|
| Title: | Comparing Fuzzers on a Level Playing Field with FuzzBench |
| Event: | International Conference on Software Testing (ICST) |
| Location: | Online |
| Dates: | 4 Apr 2022 - 13 Apr 2022 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://ieeexplore.ieee.org/Xplore/home.jsp |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Fuzzing, Software Testing, FuzzBench, Empirical Study |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10144606 |
Archive Staff Only
 |
View Item |
