Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929509576474624 |
|---|---|
| author | Watson, Julia Lee, Sophia Beekhuizen, Barend Stevenson, Suzanne |
| author_facet | Watson, Julia Lee, Sophia Beekhuizen, Barend Stevenson, Suzanne |
| contents | We study language ideologies in text produced by LLMs through a case study on English gendered language reform (related to role nouns like congressperson/-woman/-man, and singular they). First, we find political bias: when asked to use language that is "correct" or "natural", LLMs use language most similarly to when asked to align with conservative (vs. progressive) values. This shows how LLMs' metalinguistic preferences can implicitly communicate the language ideologies of a particular political group, even in seemingly non-political contexts. Second, we find LLMs exhibit internal inconsistency: LLMs use gender-neutral variants more often when more explicit metalinguistic context is provided. This shows how the language ideologies expressed in text produced by LLMs can vary, which may be unexpected to users. We discuss the broader implications of these findings for value alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_13852 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs Watson, Julia Lee, Sophia Beekhuizen, Barend Stevenson, Suzanne Computation and Language Artificial Intelligence We study language ideologies in text produced by LLMs through a case study on English gendered language reform (related to role nouns like congressperson/-woman/-man, and singular they). First, we find political bias: when asked to use language that is "correct" or "natural", LLMs use language most similarly to when asked to align with conservative (vs. progressive) values. This shows how LLMs' metalinguistic preferences can implicitly communicate the language ideologies of a particular political group, even in seemingly non-political contexts. Second, we find LLMs exhibit internal inconsistency: LLMs use gender-neutral variants more often when more explicit metalinguistic context is provided. This shows how the language ideologies expressed in text produced by LLMs can vary, which may be unexpected to users. We discuss the broader implications of these findings for value alignment. |
| title | Do language models practice what they preach? Examining language ideologies about gendered language reform encoded in LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.13852 |