Are LLMs Good Safety Agents or a Propaganda Engine?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918222236745728 |
|---|---|
| author | Yadav, Neemesh Ortu, Francesco Liu, Jiarui Yook, Joeun Schölkopf, Bernhard Mihalcea, Rada Cazzaniga, Alberto Jin, Zhijing |
| author_facet | Yadav, Neemesh Ortu, Francesco Liu, Jiarui Yook, Joeun Schölkopf, Bernhard Mihalcea, Rada Cazzaniga, Alberto Jin, Zhijing |
| contents | Large Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies or an indication of political censorship, that is practiced globally by countries, is lacking. Differentiating between safety influenced refusals or politically motivated censorship is hard and unclear. For this purpose we introduce PSP, a dataset built specifically to probe the refusal behaviors in LLMs from an explicitly political context. PSP is built by formatting existing censored content from two data sources, openly available on the internet: sensitive prompts in China generalized to multiple countries, and tweets that have been censored in various countries. We study: 1) impact of political sensitivity in seven LLMs through data-driven (making PSP implicit) and representation-level approaches (erasing the concept of politics); and, 2) vulnerability of models on PSP through prompt injection attacks (PIAs). Associating censorship with refusals on content with masked implicit intent, we find that most LLMs perform some form of censorship. We conclude with summarizing major attributes that can cause a shift in refusal distributions across models and contexts of different countries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_23174 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Are LLMs Good Safety Agents or a Propaganda Engine? Yadav, Neemesh Ortu, Francesco Liu, Jiarui Yook, Joeun Schölkopf, Bernhard Mihalcea, Rada Cazzaniga, Alberto Jin, Zhijing Computation and Language Large Language Models (LLMs) are trained to refuse to respond to harmful content. However, systematic analyses of whether this behavior is truly a reflection of its safety policies or an indication of political censorship, that is practiced globally by countries, is lacking. Differentiating between safety influenced refusals or politically motivated censorship is hard and unclear. For this purpose we introduce PSP, a dataset built specifically to probe the refusal behaviors in LLMs from an explicitly political context. PSP is built by formatting existing censored content from two data sources, openly available on the internet: sensitive prompts in China generalized to multiple countries, and tweets that have been censored in various countries. We study: 1) impact of political sensitivity in seven LLMs through data-driven (making PSP implicit) and representation-level approaches (erasing the concept of politics); and, 2) vulnerability of models on PSP through prompt injection attacks (PIAs). Associating censorship with refusals on content with masked implicit intent, we find that most LLMs perform some form of censorship. We conclude with summarizing major attributes that can cause a shift in refusal distributions across models and contexts of different countries. |
| title | Are LLMs Good Safety Agents or a Propaganda Engine? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.23174 |