Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.07678 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909897613901824 |
|---|---|
| author | Alur, Rohan Stadie, Bradly C. Kang, Daniel Chen, Ryan McManus, Matt Rickert, Michael Lee, Tyler Federici, Michael Zhu, Richard Fogerty, Dennis Williamson, Hayley Lozinski, Nina Linsky, Aaron Sekhon, Jasjeet S. |
| author_facet | Alur, Rohan Stadie, Bradly C. Kang, Daniel Chen, Ryan McManus, Matt Rickert, Michael Lee, Tyler Federici, Michael Zhu, Richard Fogerty, Dennis Williamson, Hayley Lozinski, Nina Linsky, Aaron Sekhon, Jasjeet S. |
| contents | This technical report describes the AIA Forecaster, a Large Language Model (LLM)-based system for judgmental forecasting using unstructured data. The AIA Forecaster approach combines three core elements: agentic search over high-quality news sources, a supervisor agent that reconciles disparate forecasts for the same event, and a set of statistical calibration techniques to counter behavioral biases in large language models. On the ForecastBench benchmark (Karger et al., 2024), the AIA Forecaster achieves performance equal to human superforecasters, surpassing prior LLM baselines. In addition to reporting on ForecastBench, we also introduce a more challenging forecasting benchmark sourced from liquid prediction markets. While the AIA Forecaster underperforms market consensus on this benchmark, an ensemble combining AIA Forecaster with market consensus outperforms consensus alone, demonstrating that our forecaster provides additive information. Our work establishes a new state of the art in AI forecasting and provides practical, transferable recommendations for future research. To the best of our knowledge, this is the first work that verifiably achieves expert-level forecasting at scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_07678 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AIA Forecaster: Technical Report Alur, Rohan Stadie, Bradly C. Kang, Daniel Chen, Ryan McManus, Matt Rickert, Michael Lee, Tyler Federici, Michael Zhu, Richard Fogerty, Dennis Williamson, Hayley Lozinski, Nina Linsky, Aaron Sekhon, Jasjeet S. Artificial Intelligence This technical report describes the AIA Forecaster, a Large Language Model (LLM)-based system for judgmental forecasting using unstructured data. The AIA Forecaster approach combines three core elements: agentic search over high-quality news sources, a supervisor agent that reconciles disparate forecasts for the same event, and a set of statistical calibration techniques to counter behavioral biases in large language models. On the ForecastBench benchmark (Karger et al., 2024), the AIA Forecaster achieves performance equal to human superforecasters, surpassing prior LLM baselines. In addition to reporting on ForecastBench, we also introduce a more challenging forecasting benchmark sourced from liquid prediction markets. While the AIA Forecaster underperforms market consensus on this benchmark, an ensemble combining AIA Forecaster with market consensus outperforms consensus alone, demonstrating that our forecaster provides additive information. Our work establishes a new state of the art in AI forecasting and provides practical, transferable recommendations for future research. To the best of our knowledge, this is the first work that verifiably achieves expert-level forecasting at scale. |
| title | AIA Forecaster: Technical Report |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2511.07678 |