FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916739117219840 |
|---|---|
| author | Yuan, Zhangdie Ding, Zifeng Vlachos, Andreas |
| author_facet | Yuan, Zhangdie Ding, Zifeng Vlachos, Andreas |
| contents | Forecasting is an important task in many domains, such as technology and economics. However existing forecasting benchmarks largely lack comprehensive confidence assessment, focus on limited question types, and often consist of artificial questions that do not align with real-world human forecasting needs. To address these gaps, we introduce FOReCAst (Future Outcome Reasoning and Confidence Assessment), a benchmark that evaluates models' ability to make predictions and their confidence in them. FOReCAst spans diverse forecasting scenarios involving Boolean questions, timeframe prediction, and quantity estimation, enabling a comprehensive evaluation of both prediction accuracy and confidence calibration for real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_19676 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark Yuan, Zhangdie Ding, Zifeng Vlachos, Andreas Machine Learning Computation and Language Forecasting is an important task in many domains, such as technology and economics. However existing forecasting benchmarks largely lack comprehensive confidence assessment, focus on limited question types, and often consist of artificial questions that do not align with real-world human forecasting needs. To address these gaps, we introduce FOReCAst (Future Outcome Reasoning and Confidence Assessment), a benchmark that evaluates models' ability to make predictions and their confidence in them. FOReCAst spans diverse forecasting scenarios involving Boolean questions, timeframe prediction, and quantity estimation, enabling a comprehensive evaluation of both prediction accuracy and confidence calibration for real-world applications. |
| title | FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2502.19676 |