FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipeline
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917825253212160 |
|---|---|
| author | Singh, Kuldeep Kaur, Simerjot Smiley, Charese |
| author_facet | Singh, Kuldeep Kaur, Simerjot Smiley, Charese |
| contents | Financial decision-making hinges on the analysis of relevant information embedded in the enormous volume of documents in the financial domain. To address this challenge, we developed FinQAPT, an end-to-end pipeline that streamlines the identification of relevant financial reports based on a query, extracts pertinent context, and leverages Large Language Models (LLMs) to perform downstream tasks. To evaluate the pipeline, we experimented with various techniques to optimize the performance of each module using the FinQA dataset. We introduced a novel clustering-based negative sampling technique to enhance context extraction and a novel prompting method called Dynamic N-shot Prompting to boost the numerical question-answering capabilities of LLMs. At the module level, we achieved state-of-the-art accuracy on FinQA, attaining an accuracy of 80.6%. However, at the pipeline level, we observed decreased performance due to challenges in extracting relevant context from financial reports. We conducted a detailed error analysis of each module and the end-to-end pipeline, pinpointing specific challenges that must be addressed to develop a robust solution for handling complex financial tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13959 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipeline Singh, Kuldeep Kaur, Simerjot Smiley, Charese Information Retrieval Artificial Intelligence Computation and Language Machine Learning I.2.7; H.3.3; I.2.6; I.5.3 Financial decision-making hinges on the analysis of relevant information embedded in the enormous volume of documents in the financial domain. To address this challenge, we developed FinQAPT, an end-to-end pipeline that streamlines the identification of relevant financial reports based on a query, extracts pertinent context, and leverages Large Language Models (LLMs) to perform downstream tasks. To evaluate the pipeline, we experimented with various techniques to optimize the performance of each module using the FinQA dataset. We introduced a novel clustering-based negative sampling technique to enhance context extraction and a novel prompting method called Dynamic N-shot Prompting to boost the numerical question-answering capabilities of LLMs. At the module level, we achieved state-of-the-art accuracy on FinQA, attaining an accuracy of 80.6%. However, at the pipeline level, we observed decreased performance due to challenges in extracting relevant context from financial reports. We conducted a detailed error analysis of each module and the end-to-end pipeline, pinpointing specific challenges that must be addressed to develop a robust solution for handling complex financial tasks. |
| title | FinQAPT: Empowering Financial Decisions with End-to-End LLM-driven Question Answering Pipeline |
| topic | Information Retrieval Artificial Intelligence Computation and Language Machine Learning I.2.7; H.3.3; I.2.6; I.5.3 |
| url | https://arxiv.org/abs/2410.13959 |