Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.07920 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914082467086336 |
|---|---|
| author | Li, Xiangyu Zeng, Yawen Xing, Xiaofen Xu, Jin Xu, Xiangmin |
| author_facet | Li, Xiangyu Zeng, Yawen Xing, Xiaofen Xu, Jin Xu, Xiangmin |
| contents | LLM-based financial agents have attracted widespread excitement for their ability to trade like human experts. However, most systems exhibit a "profit mirage": dazzling back-tested returns evaporate once the model's knowledge window ends, because of the inherent information leakage in LLMs. In this paper, we systematically quantify this leakage issue across four dimensions and release FinLake-Bench, a leakage-robust evaluation benchmark. Furthermore, to mitigate this issue, we introduce FactFin, a framework that applies counterfactual perturbations to compel LLM-based agents to learn causal drivers instead of memorized outcomes. FactFin integrates four core components: Strategy Code Generator, Retrieval-Augmented Generation, Monte Carlo Tree Search, and Counterfactual Simulator. Extensive experiments show that our method surpasses all baselines in out-of-sample generalization, delivering superior risk-adjusted performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07920 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Profit Mirage: Revisiting Information Leakage in LLM-based Financial Agents Li, Xiangyu Zeng, Yawen Xing, Xiaofen Xu, Jin Xu, Xiangmin Artificial Intelligence LLM-based financial agents have attracted widespread excitement for their ability to trade like human experts. However, most systems exhibit a "profit mirage": dazzling back-tested returns evaporate once the model's knowledge window ends, because of the inherent information leakage in LLMs. In this paper, we systematically quantify this leakage issue across four dimensions and release FinLake-Bench, a leakage-robust evaluation benchmark. Furthermore, to mitigate this issue, we introduce FactFin, a framework that applies counterfactual perturbations to compel LLM-based agents to learn causal drivers instead of memorized outcomes. FactFin integrates four core components: Strategy Code Generator, Retrieval-Augmented Generation, Monte Carlo Tree Search, and Counterfactual Simulator. Extensive experiments show that our method surpasses all baselines in out-of-sample generalization, delivering superior risk-adjusted performance. |
| title | Profit Mirage: Revisiting Information Leakage in LLM-based Financial Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.07920 |