A Test of Lookahead Bias in LLM Forecasts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914224992681984 |
|---|---|
| author | Gao, Zhenyu Jiang, Wenxi Yan, Yutong |
| author_facet | Gao, Zhenyu Jiang, Wenxi Yan, Yutong |
| contents | We develop a statistical test to detect lookahead bias in economic forecasts generated by large language models (LLMs). Using state-of-the-art pre-training data detection techniques, we estimate the likelihood that a given prompt appeared in an LLM's training corpus, a statistic we term Lookahead Propensity (LAP). We formally show that a positive correlation between LAP and forecast accuracy indicates the presence and magnitude of lookahead bias, and apply the test to two forecasting tasks: news headlines predicting stock returns and earnings call transcripts predicting capital expenditures. Our test provides a cost-efficient, diagnostic tool for assessing the validity and reliability of LLM-generated forecasts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_23847 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Test of Lookahead Bias in LLM Forecasts Gao, Zhenyu Jiang, Wenxi Yan, Yutong General Finance Machine Learning Trading and Market Microstructure We develop a statistical test to detect lookahead bias in economic forecasts generated by large language models (LLMs). Using state-of-the-art pre-training data detection techniques, we estimate the likelihood that a given prompt appeared in an LLM's training corpus, a statistic we term Lookahead Propensity (LAP). We formally show that a positive correlation between LAP and forecast accuracy indicates the presence and magnitude of lookahead bias, and apply the test to two forecasting tasks: news headlines predicting stock returns and earnings call transcripts predicting capital expenditures. Our test provides a cost-efficient, diagnostic tool for assessing the validity and reliability of LLM-generated forecasts. |
| title | A Test of Lookahead Bias in LLM Forecasts |
| topic | General Finance Machine Learning Trading and Market Microstructure |
| url | https://arxiv.org/abs/2512.23847 |