Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910216674607104 |
|---|---|
| author | Stähelin, Lukas Solopova, Veronika Upravitelev, Max Kaplan, David Sahitaj, Ariana Sahitaj, Premtim Jakob, Charlott Möller, Sebastian Schmitt, Vera |
| author_facet | Stähelin, Lukas Solopova, Veronika Upravitelev, Max Kaplan, David Sahitaj, Ariana Sahitaj, Premtim Jakob, Charlott Möller, Sebastian Schmitt, Vera |
| contents | Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements. We introduce a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema. Along three dimensions (model portfolio, schema effects, and prompting strategy) we evaluate the taxonomies as a classification task with the help of four language models (GPT-4.1-nano, Phi-4 14B, Qwen2.5-14B, Qwen3-14B). Our results show that fine-tuning is essential, since it transforms weak zero-shot baselines into competitive systems and reveals methodological differences that are hidden using base models. Across schemas, the Qwen models achieve the strongest overall performance, and Phi-4 14B consistently outperforms GPT-4.1-nano. Our hierarchical prompting method (HiPP), which predicts fine-grained techniques before aggregating them, is especially beneficial after fine-tuning and on the more ambiguous, low-agreement taxonomy, while remaining competitive on the simpler schema. The HQP dataset, annotated with the new intent-based labels, provides a richer lens on propaganda's strategic goals and a challenging benchmark for future work on robust, real-world detection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_13663 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas Stähelin, Lukas Solopova, Veronika Upravitelev, Max Kaplan, David Sahitaj, Ariana Sahitaj, Premtim Jakob, Charlott Möller, Sebastian Schmitt, Vera Computation and Language Computers and Society Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements. We introduce a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema. Along three dimensions (model portfolio, schema effects, and prompting strategy) we evaluate the taxonomies as a classification task with the help of four language models (GPT-4.1-nano, Phi-4 14B, Qwen2.5-14B, Qwen3-14B). Our results show that fine-tuning is essential, since it transforms weak zero-shot baselines into competitive systems and reveals methodological differences that are hidden using base models. Across schemas, the Qwen models achieve the strongest overall performance, and Phi-4 14B consistently outperforms GPT-4.1-nano. Our hierarchical prompting method (HiPP), which predicts fine-grained techniques before aggregating them, is especially beneficial after fine-tuning and on the more ambiguous, low-agreement taxonomy, while remaining competitive on the simpler schema. The HQP dataset, annotated with the new intent-based labels, provides a richer lens on propaganda's strategic goals and a challenging benchmark for future work on robust, real-world detection. |
| title | Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas |
| topic | Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2605.13663 |