"The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915074743992320 |
|---|---|
| author | Fu, Yu Guo, Shunan Hoffswell, Jane Bursztyn, Victor S. Rossi, Ryan Stasko, John |
| author_facet | Fu, Yu Guo, Shunan Hoffswell, Jane Bursztyn, Victor S. Rossi, Ryan Stasko, John |
| contents | Fact-checking data claims requires data evidence retrieval and analysis, which can become tedious and intractable when done manually. This work presents Aletheia, an automated fact-checking prototype designed to facilitate data claims verification and enhance data evidence communication. For verification, we utilize a pre-trained LLM to parse the semantics for evidence retrieval. To effectively communicate the data evidence, we design representations in two forms: data tables and visualizations, tailored to various data fact types. Additionally, we design interactions that showcase a real-world application of these techniques. We evaluate the performance of two core NLP tasks with a curated dataset comprising 400 data claims and compare the two representation forms regarding viewers' assessment time, confidence, and preference via a user study with 20 participants. The evaluation offers insights into the feasibility and bottlenecks of using LLMs for data fact-checking tasks, potential advantages and disadvantages of using visualizations over data tables, and design recommendations for presenting data evidence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10713 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | "The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims Fu, Yu Guo, Shunan Hoffswell, Jane Bursztyn, Victor S. Rossi, Ryan Stasko, John Human-Computer Interaction H.5.2; I.7.2; I.2.7 Fact-checking data claims requires data evidence retrieval and analysis, which can become tedious and intractable when done manually. This work presents Aletheia, an automated fact-checking prototype designed to facilitate data claims verification and enhance data evidence communication. For verification, we utilize a pre-trained LLM to parse the semantics for evidence retrieval. To effectively communicate the data evidence, we design representations in two forms: data tables and visualizations, tailored to various data fact types. Additionally, we design interactions that showcase a real-world application of these techniques. We evaluate the performance of two core NLP tasks with a curated dataset comprising 400 data claims and compare the two representation forms regarding viewers' assessment time, confidence, and preference via a user study with 20 participants. The evaluation offers insights into the feasibility and bottlenecks of using LLMs for data fact-checking tasks, potential advantages and disadvantages of using visualizations over data tables, and design recommendations for presenting data evidence. |
| title | "The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims |
| topic | Human-Computer Interaction H.5.2; I.7.2; I.2.7 |
| url | https://arxiv.org/abs/2409.10713 |