PreScience: A Benchmark for Forecasting Scientific Contributions
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911464300740608 |
|---|---|
| author | Ajith, Anirudh Singh, Amanpreet DeYoung, Jay Kunievsky, Nadav Kozlowski, Austin C. Tafjord, Oyvind Evans, James Weld, Daniel S. Hope, Tom Downey, Doug |
| author_facet | Ajith, Anirudh Singh, Amanpreet DeYoung, Jay Kunievsky, Nadav Kozlowski, Austin C. Tafjord, Oyvind Evans, James Weld, Daniel S. Hope, Tom Downey, Doug |
| contents | Can AI systems trained on the scientific record up to a fixed point in time forecast the scientific advances that follow? Such a capability could help researchers identify collaborators and impactful research directions, and anticipate which problems and methods will become central next. We introduce PreScience -- a scientific forecasting benchmark that decomposes the research process into four interdependent generative tasks: collaborator prediction, prior work selection, contribution generation, and impact prediction. PreScience is a carefully curated dataset of 98K recent AI-related research papers, featuring disambiguated author identities, temporally aligned scholarly metadata, and a structured graph of companion author publication histories and citations spanning 502K total papers. We develop baselines and evaluations for each task, including LACERScore, a novel LLM-based measure of contribution similarity that outperforms previous metrics and approximates inter-annotator agreement. We find substantial headroom remains in each task -- e.g. in contribution generation, frontier LLMs achieve only moderate similarity to the ground-truth (GPT-5, averages 5.6 on a 1-10 scale). When composed into a 12-month end-to-end simulation of scientific production, the resulting synthetic corpus is systematically less diverse and less novel than human-authored research from the same period. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_20459 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PreScience: A Benchmark for Forecasting Scientific Contributions Ajith, Anirudh Singh, Amanpreet DeYoung, Jay Kunievsky, Nadav Kozlowski, Austin C. Tafjord, Oyvind Evans, James Weld, Daniel S. Hope, Tom Downey, Doug Artificial Intelligence Computation and Language Can AI systems trained on the scientific record up to a fixed point in time forecast the scientific advances that follow? Such a capability could help researchers identify collaborators and impactful research directions, and anticipate which problems and methods will become central next. We introduce PreScience -- a scientific forecasting benchmark that decomposes the research process into four interdependent generative tasks: collaborator prediction, prior work selection, contribution generation, and impact prediction. PreScience is a carefully curated dataset of 98K recent AI-related research papers, featuring disambiguated author identities, temporally aligned scholarly metadata, and a structured graph of companion author publication histories and citations spanning 502K total papers. We develop baselines and evaluations for each task, including LACERScore, a novel LLM-based measure of contribution similarity that outperforms previous metrics and approximates inter-annotator agreement. We find substantial headroom remains in each task -- e.g. in contribution generation, frontier LLMs achieve only moderate similarity to the ground-truth (GPT-5, averages 5.6 on a 1-10 scale). When composed into a 12-month end-to-end simulation of scientific production, the resulting synthetic corpus is systematically less diverse and less novel than human-authored research from the same period. |
| title | PreScience: A Benchmark for Forecasting Scientific Contributions |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2602.20459 |