Hort, M;
Zhang, J;
Sarro, F;
Harman, M;
(2021)
Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods.
In:
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
(pp. pp. 994-1006).
Association for Computing Machinery: New York, NY, United States.
(In press).
Preview |
Text
Fairea-rps.pdf - Accepted Version Download (1MB) | Preview |
Abstract
The increasingly wide uptake of Machine Learning (ML) has raised the significance of the problem of tackling bias (i.e., unfairness), making it a primary software engineering concern. In this paper, we introduce Fairea, a model behaviour mutation approach to benchmarking ML bias mitigation methods. We also report on a large-scale empirical study to test the effectiveness of 12 widely-studied bias mitigation methods. Our results reveal that, surprisingly, bias mitigation methods have a poor effectiveness in 49% of the cases. In particular, 15% of the mitigation cases have worse fairness-accuracy trade-offs than the baseline established by Fairea; 34% of the cases have a decrease in accuracy and an increase in bias. Fairea has been made publicly available for software engineers and researchers to evaluate their bias mitigation methods.
| Type: | Proceedings paper |
|---|---|
| Title: | Fairea: A Model Behaviour Mutation Approach to Benchmarking Bias Mitigation Methods |
| Event: | ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.6084/m9.figshare.13712827.v2 |
| Publisher version: | https://doi.org/10.6084/m9.figshare.13712827.v2 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10130186 |
Loading...
Loading...Loading...Loading...Archive Staff Only
 |
View Item |
