Fine-tuning Smaller Language Models for Question Answering over Financial Documents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911999512805376 |
|---|---|
| author | Phogat, Karmvir Singh Puranam, Sai Akhil Dasaratha, Sridhar Harsha, Chetan Ramakrishna, Shashishekar |
| author_facet | Phogat, Karmvir Singh Puranam, Sai Akhil Dasaratha, Sridhar Harsha, Chetan Ramakrishna, Shashishekar |
| contents | Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. We explore this paradigm for the financial domain, focusing on the challenge of answering questions that require multi-hop numerical reasoning over financial texts. We assess the performance of several smaller models that have been fine-tuned to generate programs that encode the required financial reasoning and calculations. Our findings demonstrate that these fine-tuned smaller models approach the performance of the teacher model.
To provide a granular analysis of model performance, we propose an approach to investigate the specific student model capabilities that are enhanced by fine-tuning. Our empirical analysis indicates that fine-tuning refines the student models ability to express and apply the required financial concepts along with adapting the entity extraction for the specific data format. In addition, we hypothesize and demonstrate that comparable financial reasoning capability can be induced using relatively smaller datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_12337 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fine-tuning Smaller Language Models for Question Answering over Financial Documents Phogat, Karmvir Singh Puranam, Sai Akhil Dasaratha, Sridhar Harsha, Chetan Ramakrishna, Shashishekar Computation and Language Artificial Intelligence Machine Learning Systems and Control Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. We explore this paradigm for the financial domain, focusing on the challenge of answering questions that require multi-hop numerical reasoning over financial texts. We assess the performance of several smaller models that have been fine-tuned to generate programs that encode the required financial reasoning and calculations. Our findings demonstrate that these fine-tuned smaller models approach the performance of the teacher model. To provide a granular analysis of model performance, we propose an approach to investigate the specific student model capabilities that are enhanced by fine-tuning. Our empirical analysis indicates that fine-tuning refines the student models ability to express and apply the required financial concepts along with adapting the entity extraction for the specific data format. In addition, we hypothesize and demonstrate that comparable financial reasoning capability can be induced using relatively smaller datasets. |
| title | Fine-tuning Smaller Language Models for Question Answering over Financial Documents |
| topic | Computation and Language Artificial Intelligence Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2408.12337 |