Chen, Zhenpeng; Zhang, Jie M; Sarro, Federica; Harman, Mark; (2024) Fairness Improvement with Multiple Protected Attributes: How Far Are We? In: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). IEEE/ACM: Lisbon, Portugal. (In press).

Preview

Text Chen_ICSE24_Multi_attribtue_Fairness.pdf Download (1MB) | Preview

Abstract

Existing research mostly improves the fairness of Machine Learning (ML) software regarding a single protected attribute at a time, but this is unrealistic given that many users have multiple protected attributes. This paper conducts an extensive study of fairness improvement regarding multiple protected attributes, covering 11 state-of-the-art fairness improvement methods. We analyze the effectiveness of these methods with different datasets, metrics, and ML models when considering multiple protected attributes. The results reveal that improving fairness for a single protected attribute can largely decrease fairness regarding unconsidered protected attributes. This decrease is observed in up to 88.3% of scenarios (57.5% on average). More surprisingly, we find little difference in accuracy loss when considering single and multiple protected attributes, indicating that accuracy can be maintained in the multiple-attribute paradigm. However, the effect on F1-score when handling two protected attributes is about twice that of a single attribute. This has important implications for future fairness research: reporting only accuracy as the ML performance metric, which is currently common in the literature, is inadequate.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as CSVJSONXML

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item