Fairness Auditing with Multi-Agent Collaboration
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909289676800000 |
|---|---|
| author | de Vos, Martijn Dhasade, Akash Bourrée, Jade Garcia Kermarrec, Anne-Marie Merrer, Erwan Le Rottembourg, Benoit Tredan, Gilles |
| author_facet | de Vos, Martijn Dhasade, Akash Bourrée, Jade Garcia Kermarrec, Anne-Marie Merrer, Erwan Le Rottembourg, Benoit Tredan, Gilles |
| contents | Existing work in fairness auditing assumes that each audit is performed independently. In this paper, we consider multiple agents working together, each auditing the same platform for different tasks. Agents have two levers: their collaboration strategy, with or without coordination beforehand, and their strategy for sampling appropriate data points. We theoretically compare the interplay of these levers. Our main findings are that (i) collaboration is generally beneficial for accurate audits, (ii) basic sampling methods often prove to be effective, and (iii) counter-intuitively, extensive coordination on queries often deteriorates audits accuracy as the number of agents increases. Experiments on three large datasets confirm our theoretical results. Our findings motivate collaboration during fairness audits of platforms that use ML models for decision-making. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_08522 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fairness Auditing with Multi-Agent Collaboration de Vos, Martijn Dhasade, Akash Bourrée, Jade Garcia Kermarrec, Anne-Marie Merrer, Erwan Le Rottembourg, Benoit Tredan, Gilles Machine Learning Existing work in fairness auditing assumes that each audit is performed independently. In this paper, we consider multiple agents working together, each auditing the same platform for different tasks. Agents have two levers: their collaboration strategy, with or without coordination beforehand, and their strategy for sampling appropriate data points. We theoretically compare the interplay of these levers. Our main findings are that (i) collaboration is generally beneficial for accurate audits, (ii) basic sampling methods often prove to be effective, and (iii) counter-intuitively, extensive coordination on queries often deteriorates audits accuracy as the number of agents increases. Experiments on three large datasets confirm our theoretical results. Our findings motivate collaboration during fairness audits of platforms that use ML models for decision-making. |
| title | Fairness Auditing with Multi-Agent Collaboration |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.08522 |