RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909987361521664 |
|---|---|
| author | James, Joseph Xiao, Chenghao Li, Yucheng Moosavi, Nafise Sadat Lin, Chenghua |
| author_facet | James, Joseph Xiao, Chenghao Li, Yucheng Moosavi, Nafise Sadat Lin, Chenghua |
| contents | Scientific rigour tends to be sidelined in favour of bold statements, leading authors to overstate claims beyond what their results support. We present RIGOURATE, a two-stage multimodal framework that retrieves supporting evidence from a paper's body and assigns each claim an overstatement score. The framework consists of a dataset of over 10K claim-evidence sets from ICLR and NeurIPS papers, annotated using eight LLMs, with overstatement scores calibrated using peer-review comments and validated through human evaluation. It employes a fine-tuned reranker for evidence retrieval and a fine-tuned model to predict overstatement scores with justification. Compared to strong baselines, RIGOURATE enables improved evidence retrieval and overstatement detection. Overall, our work operationalises evidential proportionality and supports clearer, more transparent scientific communication. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04350 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation James, Joseph Xiao, Chenghao Li, Yucheng Moosavi, Nafise Sadat Lin, Chenghua Computation and Language Scientific rigour tends to be sidelined in favour of bold statements, leading authors to overstate claims beyond what their results support. We present RIGOURATE, a two-stage multimodal framework that retrieves supporting evidence from a paper's body and assigns each claim an overstatement score. The framework consists of a dataset of over 10K claim-evidence sets from ICLR and NeurIPS papers, annotated using eight LLMs, with overstatement scores calibrated using peer-review comments and validated through human evaluation. It employes a fine-tuned reranker for evidence retrieval and a fine-tuned model to predict overstatement scores with justification. Compared to strong baselines, RIGOURATE enables improved evidence retrieval and overstatement detection. Overall, our work operationalises evidential proportionality and supports clearer, more transparent scientific communication. |
| title | RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.04350 |