PolBiX: Detecting LLMs' Political Bias in Fact-Checking through X-phemisms
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908554292625408 |
|---|---|
| author | Jakob, Charlott Harbecke, David Parschan, Patrick Neves, Pia Wenzel Schmitt, Vera |
| author_facet | Jakob, Charlott Harbecke, David Parschan, Patrick Neves, Pia Wenzel Schmitt, Vera |
| contents | Large Language Models are increasingly used in applications requiring objective assessment, which could be compromised by political bias. Many studies found preferences for left-leaning positions in LLMs, but downstream effects on tasks like fact-checking remain underexplored. In this study, we systematically investigate political bias through exchanging words with euphemisms or dysphemisms in German claims. We construct minimal pairs of factually equivalent claims that differ in political connotation, to assess the consistency of LLMs in classifying them as true or false. We evaluate six LLMs and find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment. While a few models show tendencies of political bias, this is not mitigated by explicitly calling for objectivism in prompts. Warning: This paper contains content that may be offensive or upsetting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_15335 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PolBiX: Detecting LLMs' Political Bias in Fact-Checking through X-phemisms Jakob, Charlott Harbecke, David Parschan, Patrick Neves, Pia Wenzel Schmitt, Vera Computation and Language Large Language Models are increasingly used in applications requiring objective assessment, which could be compromised by political bias. Many studies found preferences for left-leaning positions in LLMs, but downstream effects on tasks like fact-checking remain underexplored. In this study, we systematically investigate political bias through exchanging words with euphemisms or dysphemisms in German claims. We construct minimal pairs of factually equivalent claims that differ in political connotation, to assess the consistency of LLMs in classifying them as true or false. We evaluate six LLMs and find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment. While a few models show tendencies of political bias, this is not mitigated by explicitly calling for objectivism in prompts. Warning: This paper contains content that may be offensive or upsetting. |
| title | PolBiX: Detecting LLMs' Political Bias in Fact-Checking through X-phemisms |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.15335 |