Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.20130 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914202003701760 |
|---|---|
| author | Bibal, Adrien Minton, Steven N. Khider, Deborah Gil, Yolanda |
| author_facet | Bibal, Adrien Minton, Steven N. Khider, Deborah Gil, Yolanda |
| contents | Open science initiatives seek to make research outputs more transparent, accessible, and reusable, but ensuring that published findings can be independently reproduced remains a persistent challenge. In this paper we describe an AI-driven "Reproducibility Copilot" that analyzes manuscripts, code, and supplementary materials to generate structured Jupyter Notebooks and recommendations aimed at facilitating computational, or "rote", reproducibility. Our initial results suggest that the copilot has the potential to substantially reduce reproduction time (in one case from over 30 hours to about 1 hour) while achieving high coverage of figures, tables, and results suitable for computational reproduction. The system systematically detects barriers to reproducibility, including missing values for hyperparameters, undocumented preprocessing steps, and incomplete or inaccessible datasets. Although preliminary, these findings suggest that AI tools can meaningfully reduce the burden of reproducibility efforts and contribute to more transparent and verifiable scientific communication. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20130 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AI Copilots for Reproducibility in Science: A Case Study Bibal, Adrien Minton, Steven N. Khider, Deborah Gil, Yolanda Artificial Intelligence Open science initiatives seek to make research outputs more transparent, accessible, and reusable, but ensuring that published findings can be independently reproduced remains a persistent challenge. In this paper we describe an AI-driven "Reproducibility Copilot" that analyzes manuscripts, code, and supplementary materials to generate structured Jupyter Notebooks and recommendations aimed at facilitating computational, or "rote", reproducibility. Our initial results suggest that the copilot has the potential to substantially reduce reproduction time (in one case from over 30 hours to about 1 hour) while achieving high coverage of figures, tables, and results suitable for computational reproduction. The system systematically detects barriers to reproducibility, including missing values for hyperparameters, undocumented preprocessing steps, and incomplete or inaccessible datasets. Although preliminary, these findings suggest that AI tools can meaningfully reduce the burden of reproducibility efforts and contribute to more transparent and verifiable scientific communication. |
| title | AI Copilots for Reproducibility in Science: A Case Study |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2506.20130 |