Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917587566198784 |
|---|---|
| author | Zhang, Xiao Xiang, Ruoyu Yuan, Chenhan Feng, Duanyu Han, Weiguang Lopez-Lira, Alejandro Liu, Xiao-Yang Ananiadou, Sophia Peng, Min Huang, Jimin Xie, Qianqian |
| author_facet | Zhang, Xiao Xiang, Ruoyu Yuan, Chenhan Feng, Duanyu Han, Weiguang Lopez-Lira, Alejandro Liu, Xiao-Yang Ananiadou, Sophia Peng, Min Huang, Jimin Xie, Qianqian |
| contents | Despite Spanish's pivotal role in the global finance industry, a pronounced gap exists in Spanish financial natural language processing (NLP) and application studies compared to English, especially in the era of large language models (LLMs). To bridge this gap, we unveil Toisón de Oro, the first bilingual framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English. We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks. Harnessing this, we introduce FinMA-ES, an LLM designed for bilingual financial applications. We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks. The FLARE-ES benchmark results reveal a significant multilingual performance gap and bias in existing LLMs. FinMA-ES models surpass SOTA LLMs such as GPT-4 in Spanish financial tasks, due to strategic instruction tuning and leveraging data from diverse linguistic resources, highlighting the positive impact of cross-linguistic transfer. All our datasets, models, and benchmarks have been released. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_07405 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English Zhang, Xiao Xiang, Ruoyu Yuan, Chenhan Feng, Duanyu Han, Weiguang Lopez-Lira, Alejandro Liu, Xiao-Yang Ananiadou, Sophia Peng, Min Huang, Jimin Xie, Qianqian Computation and Language Despite Spanish's pivotal role in the global finance industry, a pronounced gap exists in Spanish financial natural language processing (NLP) and application studies compared to English, especially in the era of large language models (LLMs). To bridge this gap, we unveil Toisón de Oro, the first bilingual framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English. We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks. Harnessing this, we introduce FinMA-ES, an LLM designed for bilingual financial applications. We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks. The FLARE-ES benchmark results reveal a significant multilingual performance gap and bias in existing LLMs. FinMA-ES models surpass SOTA LLMs such as GPT-4 in Spanish financial tasks, due to strategic instruction tuning and leveraging data from diverse linguistic resources, highlighting the positive impact of cross-linguistic transfer. All our datasets, models, and benchmarks have been released. |
| title | Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.07405 |