MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915340076711936 |
|---|---|
| author | Horych, Tomáš Wessel, Martin Wahle, Jan Philip Ruas, Terry Waßmuth, Jerome Greiner-Petter, André Aizawa, Akiko Gipp, Bela Spinde, Timo |
| author_facet | Horych, Tomáš Wessel, Martin Wahle, Jan Philip Ruas, Terry Waßmuth, Jerome Greiner-Petter, André Aizawa, Akiko Gipp, Bela Spinde, Timo |
| contents | Media bias detection poses a complex, multifaceted problem traditionally tackled using single-task models and small in-domain datasets, consequently lacking generalizability. To address this, we introduce MAGPIE, the first large-scale multi-task pre-training approach explicitly tailored for media bias detection. To enable pre-training at scale, we present Large Bias Mixture (LBM), a compilation of 59 bias-related tasks. MAGPIE outperforms previous approaches in media bias detection on the Bias Annotation By Experts (BABE) dataset, with a relative improvement of 3.3% F1-score. MAGPIE also performs better than previous models on 5 out of 8 tasks in the Media Bias Identification Benchmark (MBIB). Using a RoBERTa encoder, MAGPIE needs only 15% of finetuning steps compared to single-task approaches. Our evaluation shows, for instance, that tasks like sentiment and emotionality boost all learning, all tasks enhance fake news detection, and scaling tasks leads to the best results. MAGPIE confirms that MTL is a promising approach for addressing media bias detection, enhancing the accuracy and efficiency of existing models. Furthermore, LBM is the first available resource collection focused on media bias MTL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07910 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions Horych, Tomáš Wessel, Martin Wahle, Jan Philip Ruas, Terry Waßmuth, Jerome Greiner-Petter, André Aizawa, Akiko Gipp, Bela Spinde, Timo Computers and Society Computation and Language Media bias detection poses a complex, multifaceted problem traditionally tackled using single-task models and small in-domain datasets, consequently lacking generalizability. To address this, we introduce MAGPIE, the first large-scale multi-task pre-training approach explicitly tailored for media bias detection. To enable pre-training at scale, we present Large Bias Mixture (LBM), a compilation of 59 bias-related tasks. MAGPIE outperforms previous approaches in media bias detection on the Bias Annotation By Experts (BABE) dataset, with a relative improvement of 3.3% F1-score. MAGPIE also performs better than previous models on 5 out of 8 tasks in the Media Bias Identification Benchmark (MBIB). Using a RoBERTa encoder, MAGPIE needs only 15% of finetuning steps compared to single-task approaches. Our evaluation shows, for instance, that tasks like sentiment and emotionality boost all learning, all tasks enhance fake news detection, and scaling tasks leads to the best results. MAGPIE confirms that MTL is a promising approach for addressing media bias detection, enhancing the accuracy and efficiency of existing models. Furthermore, LBM is the first available resource collection focused on media bias MTL. |
| title | MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions |
| topic | Computers and Society Computation and Language |
| url | https://arxiv.org/abs/2403.07910 |