How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917708836110336 |
|---|---|
| author | Lee, Jaeyoung Lu, Ximing Hessel, Jack Brahman, Faeze Yu, Youngjae Bisk, Yonatan Choi, Yejin Gabriel, Saadia |
| author_facet | Lee, Jaeyoung Lu, Ximing Hessel, Jack Brahman, Faeze Yu, Youngjae Bisk, Yonatan Choi, Yejin Gabriel, Saadia |
| contents | Given the growing influx of misinformation across news and social media, there is a critical need for systems that can provide effective real-time verification of news claims. Large language or multimodal model based verification has been proposed to scale up online policing mechanisms for mitigating spread of false and harmful content. While these can potentially reduce burden on human fact-checkers, such efforts may be hampered by foundation model training data becoming outdated. In this work, we test the limits of improving foundation model performance without continual updating through an initial study of knowledge transfer using either existing intra- and inter- domain benchmarks or explanations generated from large language models (LLMs). We evaluate on 12 public benchmarks for fact-checking and misinformation detection as well as two other tasks relevant to content moderation -- toxicity and stance detection. Our results on two recent multi-modal fact-checking benchmarks, Mocheg and Fakeddit, indicate that knowledge transfer strategies can improve Fakeddit performance over the state-of-the-art by up to 1.7% and Mocheg performance by up to 2.9%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00369 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models Lee, Jaeyoung Lu, Ximing Hessel, Jack Brahman, Faeze Yu, Youngjae Bisk, Yonatan Choi, Yejin Gabriel, Saadia Computation and Language Given the growing influx of misinformation across news and social media, there is a critical need for systems that can provide effective real-time verification of news claims. Large language or multimodal model based verification has been proposed to scale up online policing mechanisms for mitigating spread of false and harmful content. While these can potentially reduce burden on human fact-checkers, such efforts may be hampered by foundation model training data becoming outdated. In this work, we test the limits of improving foundation model performance without continual updating through an initial study of knowledge transfer using either existing intra- and inter- domain benchmarks or explanations generated from large language models (LLMs). We evaluate on 12 public benchmarks for fact-checking and misinformation detection as well as two other tasks relevant to content moderation -- toxicity and stance detection. Our results on two recent multi-modal fact-checking benchmarks, Mocheg and Fakeddit, indicate that knowledge transfer strategies can improve Fakeddit performance over the state-of-the-art by up to 1.7% and Mocheg performance by up to 2.9%. |
| title | How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.00369 |