FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909220621778944 |
|---|---|
| author | Han, Xiaotian Chi, Jianfeng Chen, Yu Wang, Qifan Zhao, Han Zou, Na Hu, Xia |
| author_facet | Han, Xiaotian Chi, Jianfeng Chen, Yu Wang, Qifan Zhao, Han Zou, Na Hu, Xia |
| contents | This paper introduces the Fair Fairness Benchmark (\textsf{FFB}), a benchmarking framework for in-processing group fairness methods. Ensuring fairness in machine learning is important for ethical compliance. However, there exist challenges in comparing and developing fairness methods due to inconsistencies in experimental settings, lack of accessible algorithmic implementations, and limited extensibility of current fairness packages and tools. To address these issues, we introduce an open-source standardized benchmark for evaluating in-processing group fairness methods and provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness. This work offers the following key contributions: the provision of flexible, extensible, minimalistic, and research-oriented open-source code; the establishment of unified fairness method benchmarking pipelines; and extensive benchmarking, which yields key insights from $\mathbf{45,079}$ experiments, $\mathbf{14,428}$ GPU hours. We believe that our work will significantly facilitate the growth and development of the fairness research community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_09468 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods Han, Xiaotian Chi, Jianfeng Chen, Yu Wang, Qifan Zhao, Han Zou, Na Hu, Xia Machine Learning Artificial Intelligence Computers and Society This paper introduces the Fair Fairness Benchmark (\textsf{FFB}), a benchmarking framework for in-processing group fairness methods. Ensuring fairness in machine learning is important for ethical compliance. However, there exist challenges in comparing and developing fairness methods due to inconsistencies in experimental settings, lack of accessible algorithmic implementations, and limited extensibility of current fairness packages and tools. To address these issues, we introduce an open-source standardized benchmark for evaluating in-processing group fairness methods and provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness. This work offers the following key contributions: the provision of flexible, extensible, minimalistic, and research-oriented open-source code; the establishment of unified fairness method benchmarking pipelines; and extensive benchmarking, which yields key insights from $\mathbf{45,079}$ experiments, $\mathbf{14,428}$ GPU hours. We believe that our work will significantly facilitate the growth and development of the fairness research community. |
| title | FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods |
| topic | Machine Learning Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2306.09468 |