Measuring Political Bias in Large Language Models: What Is Said and How It Is Said
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909153975336960 |
|---|---|
| author | Bang, Yejin Chen, Delong Lee, Nayeon Fung, Pascale |
| author_facet | Bang, Yejin Chen, Delong Lee, Nayeon Fung, Pascale |
| contents | We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues. Existing benchmarks and measures focus on gender and racial biases. However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications. In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs. Our proposed measure looks at different political issues such as reproductive rights and climate change, at both the content (the substance of the generation) and the style (the lexical polarity) of such bias. We measured the political bias in eleven open-sourced LLMs and showed that our proposed framework is easily scalable to other topics and is explainable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_18932 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Measuring Political Bias in Large Language Models: What Is Said and How It Is Said Bang, Yejin Chen, Delong Lee, Nayeon Fung, Pascale Computation and Language Artificial Intelligence We propose to measure political bias in LLMs by analyzing both the content and style of their generated content regarding political issues. Existing benchmarks and measures focus on gender and racial biases. However, political bias exists in LLMs and can lead to polarization and other harms in downstream applications. In order to provide transparency to users, we advocate that there should be fine-grained and explainable measures of political biases generated by LLMs. Our proposed measure looks at different political issues such as reproductive rights and climate change, at both the content (the substance of the generation) and the style (the lexical polarity) of such bias. We measured the political bias in eleven open-sourced LLMs and showed that our proposed framework is easily scalable to other topics and is explainable. |
| title | Measuring Political Bias in Large Language Models: What Is Said and How It Is Said |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.18932 |