FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918162437505024 |
|---|---|
| author | Hu, Tiansheng Hu, Tongyan Bai, Liuyang Zhao, Yilun Cohan, Arman Zhao, Chen |
| author_facet | Hu, Tiansheng Hu, Tongyan Bai, Liuyang Zhao, Yilun Cohan, Arman Zhao, Chen |
| contents | Recent LLMs have demonstrated promising ability in solving finance related problems. However, applying LLMs in real-world finance application remains challenging due to its high risk and high stakes property. This paper introduces FinTrust, a comprehensive benchmark specifically designed for evaluating the trustworthiness of LLMs in finance applications. Our benchmark focuses on a wide range of alignment issues based on practical context and features fine-grained tasks for each dimension of trustworthiness evaluation. We assess eleven LLMs on FinTrust and find that proprietary models like o4-mini outperforms in most tasks such as safety while open-source models like DeepSeek-V3 have advantage in specific areas like industry-level fairness. For challenging task like fiduciary alignment and disclosure, all LLMs fall short, showing a significant gap in legal awareness. We believe that FinTrust can be a valuable benchmark for LLMs' trustworthiness evaluation in finance domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_15232 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain Hu, Tiansheng Hu, Tongyan Bai, Liuyang Zhao, Yilun Cohan, Arman Zhao, Chen Machine Learning Computation and Language Recent LLMs have demonstrated promising ability in solving finance related problems. However, applying LLMs in real-world finance application remains challenging due to its high risk and high stakes property. This paper introduces FinTrust, a comprehensive benchmark specifically designed for evaluating the trustworthiness of LLMs in finance applications. Our benchmark focuses on a wide range of alignment issues based on practical context and features fine-grained tasks for each dimension of trustworthiness evaluation. We assess eleven LLMs on FinTrust and find that proprietary models like o4-mini outperforms in most tasks such as safety while open-source models like DeepSeek-V3 have advantage in specific areas like industry-level fairness. For challenging task like fiduciary alignment and disclosure, all LLMs fall short, showing a significant gap in legal awareness. We believe that FinTrust can be a valuable benchmark for LLMs' trustworthiness evaluation in finance domain. |
| title | FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2510.15232 |