Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.18453 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918349475151872 |
|---|---|
| author | Kubota, So Yakura, Hiromu Coavoux, Samuel Yamada, Sho Nakamura, Yuki |
| author_facet | Kubota, So Yakura, Hiromu Coavoux, Samuel Yamada, Sho Nakamura, Yuki |
| contents | The replication crisis, the failure of scientific claims to be validated by further research, is one of the most pressing issues for empirical research. This is partly an incentive problem: replication is costly and less well rewarded than original research. Large language models (LLMs) have accelerated scientific production by streamlining writing, coding, and reviewing, yet this acceleration risks outpacing verification. To address this, we present an LLM-based system that replicates statistical analyses from social science papers and flags potential problems. Quantitative social science is particularly well-suited to automation because it relies on standard statistical models, shared public datasets, and uniform reporting formats such as regression tables and summary statistics. We present a prototype that iterates LLM-based text interpretation, code generation, execution, and discrepancy analysis, demonstrating its capabilities by reproducing key results from a seminal sociology paper. We also outline application scenarios including pre-submission checks, peer-review support, and meta-scientific audits, positioning AI verification as assistive infrastructure that strengthens research integrity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18453 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLM-Assisted Replication for Quantitative Social Science Kubota, So Yakura, Hiromu Coavoux, Samuel Yamada, Sho Nakamura, Yuki Computers and Society Artificial Intelligence The replication crisis, the failure of scientific claims to be validated by further research, is one of the most pressing issues for empirical research. This is partly an incentive problem: replication is costly and less well rewarded than original research. Large language models (LLMs) have accelerated scientific production by streamlining writing, coding, and reviewing, yet this acceleration risks outpacing verification. To address this, we present an LLM-based system that replicates statistical analyses from social science papers and flags potential problems. Quantitative social science is particularly well-suited to automation because it relies on standard statistical models, shared public datasets, and uniform reporting formats such as regression tables and summary statistics. We present a prototype that iterates LLM-based text interpretation, code generation, execution, and discrepancy analysis, demonstrating its capabilities by reproducing key results from a seminal sociology paper. We also outline application scenarios including pre-submission checks, peer-review support, and meta-scientific audits, positioning AI verification as assistive infrastructure that strengthens research integrity. |
| title | LLM-Assisted Replication for Quantitative Social Science |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2602.18453 |