DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted Image Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910439400538112 |
|---|---|
| author | Williams, Joshua N. FitzMorris, Molly Aka, Osman Laszlo, Sarah |
| author_facet | Williams, Joshua N. FitzMorris, Molly Aka, Osman Laszlo, Sarah |
| contents | Text-to-image models are now easy to use and ubiquitous. However, prior work has found that they are prone to recapitulating harmful Western stereotypes. For example, requesting that a model generate an "African person and their house," may produce a person standing next to a straw hut. In this example, the word "African" is an explicit descriptor of the person that the prompt is seeking to depict. Here, we examine whether implicit markers, such as dialect, can also affect the portrayal of people in text-to-image outputs. We pair prompts in Mainstream American English with counterfactuals that express grammatical constructions found in dialects correlated with historically marginalized groups. We find that through minimal, syntax-only changes to prompts, we can systematically shift the skin tone and gender of people in the generated images. We conclude with a discussion of whether dialectic distribution shifts like this are harmful or are expected, possibly even desirable, model behavior. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05382 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted Image Generation Williams, Joshua N. FitzMorris, Molly Aka, Osman Laszlo, Sarah Computers and Society Text-to-image models are now easy to use and ubiquitous. However, prior work has found that they are prone to recapitulating harmful Western stereotypes. For example, requesting that a model generate an "African person and their house," may produce a person standing next to a straw hut. In this example, the word "African" is an explicit descriptor of the person that the prompt is seeking to depict. Here, we examine whether implicit markers, such as dialect, can also affect the portrayal of people in text-to-image outputs. We pair prompts in Mainstream American English with counterfactuals that express grammatical constructions found in dialects correlated with historically marginalized groups. We find that through minimal, syntax-only changes to prompts, we can systematically shift the skin tone and gender of people in the generated images. We conclude with a discussion of whether dialectic distribution shifts like this are harmful or are expected, possibly even desirable, model behavior. |
| title | DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted Image Generation |
| topic | Computers and Society |
| url | https://arxiv.org/abs/2405.05382 |