When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912071857209344 |
|---|---|
| author | Vijay, Supriti Priyanshu, Aman KhudaBukhsh, Ashique R. |
| author_facet | Vijay, Supriti Priyanshu, Aman KhudaBukhsh, Ashique R. |
| contents | In an era where societal narratives are increasingly shaped by algorithmic curation, investigating the political neutrality of LLMs is an important research question. This study presents a fresh perspective on quantifying the political neutrality of LLMs through the lens of abstractive text summarization of polarizing news articles. We consider five pressing issues in current US politics: abortion, gun control/rights, healthcare, immigration, and LGBTQ+ rights. Via a substantial corpus of 20,344 news articles, our study reveals a consistent trend towards pro-Democratic biases in several well-known LLMs, with gun control and healthcare exhibiting the most pronounced biases (max polarization differences of -9.49% and -6.14%, respectively). Further analysis uncovers a strong convergence in the vocabulary of the LLM outputs for these divisive topics (55% overlap for Democrat-leaning representations, 52% for Republican). Being months away from a US election of consequence, we consider our findings important. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09978 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries Vijay, Supriti Priyanshu, Aman KhudaBukhsh, Ashique R. Computation and Language Computers and Society In an era where societal narratives are increasingly shaped by algorithmic curation, investigating the political neutrality of LLMs is an important research question. This study presents a fresh perspective on quantifying the political neutrality of LLMs through the lens of abstractive text summarization of polarizing news articles. We consider five pressing issues in current US politics: abortion, gun control/rights, healthcare, immigration, and LGBTQ+ rights. Via a substantial corpus of 20,344 news articles, our study reveals a consistent trend towards pro-Democratic biases in several well-known LLMs, with gun control and healthcare exhibiting the most pronounced biases (max polarization differences of -9.49% and -6.14%, respectively). Further analysis uncovers a strong convergence in the vocabulary of the LLM outputs for these divisive topics (55% overlap for Democrat-leaning representations, 52% for Republican). Being months away from a US election of consequence, we consider our findings important. |
| title | When Neutral Summaries are not that Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2410.09978 |