Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909672126021632 |
|---|---|
| author | Cheong, Inyoung Guo, Alicia Lee, Mina Liao, Zhehui Kadoma, Kowe Go, Dongyoung Chang, Joseph Chee Henderson, Peter Naaman, Mor Zhang, Amy X. |
| author_facet | Cheong, Inyoung Guo, Alicia Lee, Mina Liao, Zhehui Kadoma, Kowe Go, Dongyoung Chang, Joseph Chee Henderson, Peter Naaman, Mor Zhang, Amy X. |
| contents | As AI integrates in various types of human writing, calls for transparency around AI assistance are growing. However, if transparency operates on uneven ground and certain identity groups bear a heavier cost for being honest, then the burden of openness becomes asymmetrical. This study investigates how AI disclosure statement affects perceptions of writing quality, and whether these effects vary by the author's race and gender. Through a large-scale controlled experiment, both human raters (n = 1,970) and LLM raters (n = 2,520) evaluated a single human-written news article while disclosure statements and author demographics were systematically varied. This approach reflects how both human and algorithmic decisions now influence access to opportunities (e.g., hiring, promotion) and social recognition (e.g., content recommendation algorithms). We find that both human and LLM raters consistently penalize disclosed AI use. However, only LLM raters exhibit demographic interaction effects: they favor articles attributed to women or Black authors when no disclosure is present. But these advantages disappear when AI assistance is revealed. These findings illuminate the complex relationships between AI disclosure and author identity, highlighting disparities between machine and human evaluation patterns. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_01418 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing Cheong, Inyoung Guo, Alicia Lee, Mina Liao, Zhehui Kadoma, Kowe Go, Dongyoung Chang, Joseph Chee Henderson, Peter Naaman, Mor Zhang, Amy X. Computers and Society Artificial Intelligence H.5.2; I.2 As AI integrates in various types of human writing, calls for transparency around AI assistance are growing. However, if transparency operates on uneven ground and certain identity groups bear a heavier cost for being honest, then the burden of openness becomes asymmetrical. This study investigates how AI disclosure statement affects perceptions of writing quality, and whether these effects vary by the author's race and gender. Through a large-scale controlled experiment, both human raters (n = 1,970) and LLM raters (n = 2,520) evaluated a single human-written news article while disclosure statements and author demographics were systematically varied. This approach reflects how both human and algorithmic decisions now influence access to opportunities (e.g., hiring, promotion) and social recognition (e.g., content recommendation algorithms). We find that both human and LLM raters consistently penalize disclosed AI use. However, only LLM raters exhibit demographic interaction effects: they favor articles attributed to women or Black authors when no disclosure is present. But these advantages disappear when AI assistance is revealed. These findings illuminate the complex relationships between AI disclosure and author identity, highlighting disparities between machine and human evaluation patterns. |
| title | Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing |
| topic | Computers and Society Artificial Intelligence H.5.2; I.2 |
| url | https://arxiv.org/abs/2507.01418 |