Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2412.17052 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912918421897216 |
|---|---|
| author | Raza, Shaina Saleh, Caesar Farooq, Azib Hasan, Emrul Ogidi, Franklin Zahid, Haad Powers, Maximus Lotif, Marcelo Zahid, Anam Sekhon, Karanpal Chatrath, Veronica Javedi, Roya Khazaie, Vahid Reza Yu, Zhenyu |
| author_facet | Raza, Shaina Saleh, Caesar Farooq, Azib Hasan, Emrul Ogidi, Franklin Zahid, Haad Powers, Maximus Lotif, Marcelo Zahid, Anam Sekhon, Karanpal Chatrath, Veronica Javedi, Roya Khazaie, Vahid Reza Yu, Zhenyu |
| contents | Detecting bias in multimodal news requires models that reason over text--image pairs, not just classify text. In response, we present ViLBias, a VQA-style benchmark and framework for detecting and reasoning about bias in multimodal news. The dataset comprises 40,945 text--image pairs from diverse outlets, each annotated with a bias label and concise rationale using a two-stage LLM-as-annotator pipeline with hierarchical majority voting and human-in-the-loop validation. We evaluate Small Language Models (SLMs), Large Language Models (LLMs), and Vision--Language Models (VLMs) across closed-ended classification and open-ended reasoning (oVQA), and compare parameter-efficient tuning strategies. Results show that incorporating images alongside text improves detection accuracy by 3--5\%, and that LLMs/VLMs better capture subtle framing and text--image inconsistencies than SLMs. Parameter-efficient methods (LoRA/QLoRA/Adapters) recover 97--99\% of full fine-tuning performance with $<5\%$ trainable parameters. For oVQA, reasoning accuracy spans 52--79\% and faithfulness 68--89\%, both improved by instruction tuning; closed accuracy correlates strongly with reasoning ($r = 0.91$). ViLBias offers a scalable benchmark and strong baselines for multimodal bias detection and rationale quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17052 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ViLBias: Detecting and Reasoning about Bias in Multimodal Content Raza, Shaina Saleh, Caesar Farooq, Azib Hasan, Emrul Ogidi, Franklin Zahid, Haad Powers, Maximus Lotif, Marcelo Zahid, Anam Sekhon, Karanpal Chatrath, Veronica Javedi, Roya Khazaie, Vahid Reza Yu, Zhenyu Artificial Intelligence Detecting bias in multimodal news requires models that reason over text--image pairs, not just classify text. In response, we present ViLBias, a VQA-style benchmark and framework for detecting and reasoning about bias in multimodal news. The dataset comprises 40,945 text--image pairs from diverse outlets, each annotated with a bias label and concise rationale using a two-stage LLM-as-annotator pipeline with hierarchical majority voting and human-in-the-loop validation. We evaluate Small Language Models (SLMs), Large Language Models (LLMs), and Vision--Language Models (VLMs) across closed-ended classification and open-ended reasoning (oVQA), and compare parameter-efficient tuning strategies. Results show that incorporating images alongside text improves detection accuracy by 3--5\%, and that LLMs/VLMs better capture subtle framing and text--image inconsistencies than SLMs. Parameter-efficient methods (LoRA/QLoRA/Adapters) recover 97--99\% of full fine-tuning performance with $<5\%$ trainable parameters. For oVQA, reasoning accuracy spans 52--79\% and faithfulness 68--89\%, both improved by instruction tuning; closed accuracy correlates strongly with reasoning ($r = 0.91$). ViLBias offers a scalable benchmark and strong baselines for multimodal bias detection and rationale quality. |
| title | ViLBias: Detecting and Reasoning about Bias in Multimodal Content |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2412.17052 |