The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918067000311808 |
|---|---|
| author | Magomere, Jabez Ishida, Shu Afonja, Tejumade Salama, Aya Kochin, Daniel Yuehgoh, Foutse Hamzaoui, Imane Sefala, Raesetje Alaagib, Aisha Dalal, Samantha Marchegiani, Beatrice Semenova, Elizaveta Crais, Lauren Hall, Siobhan Mackenzie |
| author_facet | Magomere, Jabez Ishida, Shu Afonja, Tejumade Salama, Aya Kochin, Daniel Yuehgoh, Foutse Hamzaoui, Imane Sefala, Raesetje Alaagib, Aisha Dalal, Samantha Marchegiani, Beatrice Semenova, Elizaveta Crais, Lauren Hall, Siobhan Mackenzie |
| contents | We introduce the World Wide recipe, which sets forth a framework for culturally aware and participatory data collection, and the resultant regionally diverse World Wide Dishes evaluation dataset. We also analyse bias operationalisation to highlight how current systems underperform across several dimensions: (in-)accuracy, (mis-)representation, and cultural (in-)sensitivity, with evidence from qualitative community-based observations and quantitative automated tools. We find that these T2I models generally do not produce quality outputs of dishes specific to various regions. This is true even for the US, which is typically considered more well-resourced in training data -- although the generation of US dishes does outperform that of the investigated African countries. The models demonstrate the propensity to produce inaccurate and culturally misrepresentative, flattening, and insensitive outputs. These representational biases have the potential to further reinforce stereotypes and disproportionately contribute to erasure based on region. The dataset and code are available at https://github.com/oxai/world-wide-dishes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09496 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation Magomere, Jabez Ishida, Shu Afonja, Tejumade Salama, Aya Kochin, Daniel Yuehgoh, Foutse Hamzaoui, Imane Sefala, Raesetje Alaagib, Aisha Dalal, Samantha Marchegiani, Beatrice Semenova, Elizaveta Crais, Lauren Hall, Siobhan Mackenzie Computers and Society Artificial Intelligence We introduce the World Wide recipe, which sets forth a framework for culturally aware and participatory data collection, and the resultant regionally diverse World Wide Dishes evaluation dataset. We also analyse bias operationalisation to highlight how current systems underperform across several dimensions: (in-)accuracy, (mis-)representation, and cultural (in-)sensitivity, with evidence from qualitative community-based observations and quantitative automated tools. We find that these T2I models generally do not produce quality outputs of dishes specific to various regions. This is true even for the US, which is typically considered more well-resourced in training data -- although the generation of US dishes does outperform that of the investigated African countries. The models demonstrate the propensity to produce inaccurate and culturally misrepresentative, flattening, and insensitive outputs. These representational biases have the potential to further reinforce stereotypes and disproportionately contribute to erasure based on region. The dataset and code are available at https://github.com/oxai/world-wide-dishes. |
| title | The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2406.09496 |