Evaluating LLMs in Finance Requires Explicit Bias Consideration
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918339870195712 |
|---|---|
| author | Kong, Yaxuan Lee, Hoyoung Hwang, Yoontae Lopez-Lira, Alejandro Levy, Bradford Mehta, Dhagash Wen, Qingsong Choi, Chanyeol Lee, Yongjae Zohren, Stefan |
| author_facet | Kong, Yaxuan Lee, Hoyoung Hwang, Yoontae Lopez-Lira, Alejandro Levy, Bradford Mehta, Dhagash Wen, Qingsong Choi, Chanyeol Lee, Yongjae Zohren, Stefan |
| contents | Large Language Models (LLMs) are increasingly integrated into financial workflows, but evaluation practice has not kept up. Finance-specific biases can inflate performance, contaminate backtests, and make reported results useless for any deployment claim. We identify five recurring biases in financial LLM applications. They include look-ahead bias, survivorship bias, narrative bias, objective bias, and cost bias. These biases break financial tasks in distinct ways and they often compound to create an illusion of validity. We reviewed 164 papers from 2023 to 2025 and found that no single bias is discussed in more than 28 percent of studies. This position paper argues that bias in financial LLM systems requires explicit attention and that structural validity should be enforced before any result is used to support a deployment claim. We propose a Structural Validity Framework and an evaluation checklist with minimal requirements for bias diagnosis and future system design. The material is available at https://github.com/Eleanorkong/Awesome-Financial-LLM-Bias-Mitigation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_14233 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Evaluating LLMs in Finance Requires Explicit Bias Consideration Kong, Yaxuan Lee, Hoyoung Hwang, Yoontae Lopez-Lira, Alejandro Levy, Bradford Mehta, Dhagash Wen, Qingsong Choi, Chanyeol Lee, Yongjae Zohren, Stefan Machine Learning Artificial Intelligence Computational Finance Large Language Models (LLMs) are increasingly integrated into financial workflows, but evaluation practice has not kept up. Finance-specific biases can inflate performance, contaminate backtests, and make reported results useless for any deployment claim. We identify five recurring biases in financial LLM applications. They include look-ahead bias, survivorship bias, narrative bias, objective bias, and cost bias. These biases break financial tasks in distinct ways and they often compound to create an illusion of validity. We reviewed 164 papers from 2023 to 2025 and found that no single bias is discussed in more than 28 percent of studies. This position paper argues that bias in financial LLM systems requires explicit attention and that structural validity should be enforced before any result is used to support a deployment claim. We propose a Structural Validity Framework and an evaluation checklist with minimal requirements for bias diagnosis and future system design. The material is available at https://github.com/Eleanorkong/Awesome-Financial-LLM-Bias-Mitigation. |
| title | Evaluating LLMs in Finance Requires Explicit Bias Consideration |
| topic | Machine Learning Artificial Intelligence Computational Finance |
| url | https://arxiv.org/abs/2602.14233 |