FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916930527428608 |
|---|---|
| author | Choi, Chanyeol Kwon, Jihoon Ha, Jaeseon Choi, Hojun Kim, Chaewoon Lee, Yongjae Sohn, Jy-yong Lopez-Lira, Alejandro |
| author_facet | Choi, Chanyeol Kwon, Jihoon Ha, Jaeseon Choi, Hojun Kim, Chaewoon Lee, Yongjae Sohn, Jy-yong Lopez-Lira, Alejandro |
| contents | In the fast-paced financial domain, accurate and up-to-date information is critical to addressing ever-evolving market conditions. Retrieving this information correctly is essential in financial Question-Answering (QA), since many language models struggle with factual accuracy in this domain. We present FinDER, an expert-generated dataset tailored for Retrieval-Augmented Generation (RAG) in finance. Unlike existing QA datasets that provide predefined contexts and rely on relatively clear and straightforward queries, FinDER focuses on annotating search-relevant evidence by domain experts, offering 5,703 query-evidence-answer triplets derived from real-world financial inquiries. These queries frequently include abbreviations, acronyms, and concise expressions, capturing the brevity and ambiguity common in the realistic search behavior of professionals. By challenging models to retrieve relevant information from large corpora rather than relying on readily determined contexts, FinDER offers a more realistic benchmark for evaluating RAG systems. We further present a comprehensive evaluation of multiple state-of-the-art retrieval models and Large Language Models, showcasing challenges derived from a realistic benchmark to drive future research on truthful and precise RAG in the financial domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_15800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation Choi, Chanyeol Kwon, Jihoon Ha, Jaeseon Choi, Hojun Kim, Chaewoon Lee, Yongjae Sohn, Jy-yong Lopez-Lira, Alejandro Information Retrieval In the fast-paced financial domain, accurate and up-to-date information is critical to addressing ever-evolving market conditions. Retrieving this information correctly is essential in financial Question-Answering (QA), since many language models struggle with factual accuracy in this domain. We present FinDER, an expert-generated dataset tailored for Retrieval-Augmented Generation (RAG) in finance. Unlike existing QA datasets that provide predefined contexts and rely on relatively clear and straightforward queries, FinDER focuses on annotating search-relevant evidence by domain experts, offering 5,703 query-evidence-answer triplets derived from real-world financial inquiries. These queries frequently include abbreviations, acronyms, and concise expressions, capturing the brevity and ambiguity common in the realistic search behavior of professionals. By challenging models to retrieve relevant information from large corpora rather than relying on readily determined contexts, FinDER offers a more realistic benchmark for evaluating RAG systems. We further present a comprehensive evaluation of multiple state-of-the-art retrieval models and Large Language Models, showcasing challenges derived from a realistic benchmark to drive future research on truthful and precise RAG in the financial domain. |
| title | FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2504.15800 |