Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910399701450752 |
|---|---|
| author | Sundararajan, Barkavi Sripada, Somayajulu Reiter, Ehud |
| author_facet | Sundararajan, Barkavi Sripada, Somayajulu Reiter, Ehud |
| contents | Neural Table-to-Text models tend to hallucinate, producing texts that contain factual errors. We investigate whether such errors in the output can be traced back to problems with the input. We manually annotated 1,837 texts generated by multiple models in the politics domain of the ToTTo dataset. We identify the input problems that are responsible for many output errors and show that fixing these inputs reduces factual errors by between 52% and 76% (depending on the model). In addition, we observe that models struggle in processing tabular inputs that are structured in a non-standard way, particularly when the input lacks distinct row and column values or when the column headers are not correctly mapped to corresponding values. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_04103 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo Sundararajan, Barkavi Sripada, Somayajulu Reiter, Ehud Computation and Language Neural Table-to-Text models tend to hallucinate, producing texts that contain factual errors. We investigate whether such errors in the output can be traced back to problems with the input. We manually annotated 1,837 texts generated by multiple models in the politics domain of the ToTTo dataset. We identify the input problems that are responsible for many output errors and show that fixing these inputs reduces factual errors by between 52% and 76% (depending on the model). In addition, we observe that models struggle in processing tabular inputs that are structured in a non-standard way, particularly when the input lacks distinct row and column values or when the column headers are not correctly mapped to corresponding values. |
| title | Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.04103 |