"Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909863192297472 |
|---|---|
| author | Anderson, Andrew Moussaoui, Fatima A. Guevara, Jimena Noa Hamid, Md Montaser Burnett, Margaret |
| author_facet | Anderson, Andrew Moussaoui, Fatima A. Guevara, Jimena Noa Hamid, Md Montaser Burnett, Margaret |
| contents | While much research has shown the presence of AI's "under-the-hood" biases (e.g., algorithmic, training data, etc.), what about "over-the-hood" inclusivity biases: barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches? Recent research has begun to report the existence of such biases -- but what do they look like, how prevalent are they, and how can developers find and fix them? To find out, we conducted a field study with 3 AI product teams, to investigate what kinds of AI inclusivity bugs exist uniquely in user-facing AI products, and whether/how AI product teams might harness an existing (non-AI-oriented) inclusive design method to find and fix them. The teams' work resulted in identifying 6 types of AI inclusivity bugs arising 83 times, fixes covering 47 of these bug instances, and a new variation of the GenderMag inclusive design method, GenderMag-for-AI, that is especially effective at detecting certain kinds of AI inclusivity bugs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19033 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | "Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them Anderson, Andrew Moussaoui, Fatima A. Guevara, Jimena Noa Hamid, Md Montaser Burnett, Margaret Human-Computer Interaction Artificial Intelligence While much research has shown the presence of AI's "under-the-hood" biases (e.g., algorithmic, training data, etc.), what about "over-the-hood" inclusivity biases: barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches? Recent research has begun to report the existence of such biases -- but what do they look like, how prevalent are they, and how can developers find and fix them? To find out, we conducted a field study with 3 AI product teams, to investigate what kinds of AI inclusivity bugs exist uniquely in user-facing AI products, and whether/how AI product teams might harness an existing (non-AI-oriented) inclusive design method to find and fix them. The teams' work resulted in identifying 6 types of AI inclusivity bugs arising 83 times, fixes covering 47 of these bug instances, and a new variation of the GenderMag inclusive design method, GenderMag-for-AI, that is especially effective at detecting certain kinds of AI inclusivity bugs. |
| title | "Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2510.19033 |