The Automated Verification of Textual Claims (AVeriTeC) Shared Task
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913568976273408 |
|---|---|
| author | Schlichtkrull, Michael Chen, Yulong Whitehouse, Chenxi Deng, Zhenyun Akhtar, Mubashara Aly, Rami Guo, Zhijiang Christodoulopoulos, Christos Cocarascu, Oana Mittal, Arpit Thorne, James Vlachos, Andreas |
| author_facet | Schlichtkrull, Michael Chen, Yulong Whitehouse, Chenxi Deng, Zhenyun Akhtar, Mubashara Aly, Rami Guo, Zhijiang Christodoulopoulos, Christos Cocarascu, Oana Mittal, Arpit Thorne, James Vlachos, Andreas |
| contents | The Automated Verification of Textual Claims (AVeriTeC) shared task asks participants to retrieve evidence and predict veracity for real-world claims checked by fact-checkers. Evidence can be found either via a search engine, or via a knowledge store provided by the organisers. Submissions are evaluated using AVeriTeC score, which considers a claim to be accurately verified if and only if both the verdict is correct and retrieved evidence is considered to meet a certain quality threshold. The shared task received 21 submissions, 18 of which surpassed our baseline. The winning team was TUDA_MAI with an AVeriTeC score of 63%. In this paper we describe the shared task, present the full results, and highlight key takeaways from the shared task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_23850 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Automated Verification of Textual Claims (AVeriTeC) Shared Task Schlichtkrull, Michael Chen, Yulong Whitehouse, Chenxi Deng, Zhenyun Akhtar, Mubashara Aly, Rami Guo, Zhijiang Christodoulopoulos, Christos Cocarascu, Oana Mittal, Arpit Thorne, James Vlachos, Andreas Computation and Language The Automated Verification of Textual Claims (AVeriTeC) shared task asks participants to retrieve evidence and predict veracity for real-world claims checked by fact-checkers. Evidence can be found either via a search engine, or via a knowledge store provided by the organisers. Submissions are evaluated using AVeriTeC score, which considers a claim to be accurately verified if and only if both the verdict is correct and retrieved evidence is considered to meet a certain quality threshold. The shared task received 21 submissions, 18 of which surpassed our baseline. The winning team was TUDA_MAI with an AVeriTeC score of 63%. In this paper we describe the shared task, present the full results, and highlight key takeaways from the shared task. |
| title | The Automated Verification of Textual Claims (AVeriTeC) Shared Task |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.23850 |