NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914890623483904 |
|---|---|
| author | Hinterreiter, Smi Wessel, Martin Schliski, Fabian Echizen, Isao Latoschik, Marc Erich Spinde, Timo |
| author_facet | Hinterreiter, Smi Wessel, Martin Schliski, Fabian Echizen, Isao Latoschik, Marc Erich Spinde, Timo |
| contents | Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31% and improving classifier performance by 2.49%. As the first human-in-the-loop application for media bias, the feedback mechanism shows that a user-centric approach to media bias data collection can return reliable data while being scalable and evaluated as easy to use. NewsUnfold demonstrates that feedback mechanisms are a promising strategy to reduce data collection expenses and continuously update datasets to changes in context. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_17045 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback Hinterreiter, Smi Wessel, Martin Schliski, Fabian Echizen, Isao Latoschik, Marc Erich Spinde, Timo Human-Computer Interaction Media bias is a multifaceted problem, leading to one-sided views and impacting decision-making. A way to address digital media bias is to detect and indicate it automatically through machine-learning methods. However, such detection is limited due to the difficulty of obtaining reliable training data. Human-in-the-loop-based feedback mechanisms have proven an effective way to facilitate the data-gathering process. Therefore, we introduce and test feedback mechanisms for the media bias domain, which we then implement on NewsUnfold, a news-reading web application to collect reader feedback on machine-generated bias highlights within online news articles. Our approach augments dataset quality by significantly increasing inter-annotator agreement by 26.31% and improving classifier performance by 2.49%. As the first human-in-the-loop application for media bias, the feedback mechanism shows that a user-centric approach to media bias data collection can return reliable data while being scalable and evaluated as easy to use. NewsUnfold demonstrates that feedback mechanisms are a promising strategy to reduce data collection expenses and continuously update datasets to changes in context. |
| title | NewsUnfold: Creating a News-Reading Application That Indicates Linguistic Media Bias and Collects Feedback |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2407.17045 |