THaLLE-ThaiLLM: Domain-Specialized Small LLMs for Finance and Thai -- Technical Report
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908753160306688 |
|---|---|
| author | Labs, KBTG : Lertpiya, Anuruth Khamnuansin, Danupat Sucharitpongpan, Kantapong Balee, Pornchanan Chalothorn, Tawunrat Pongthawornkamol, Thadpong Lertsutthiwong, Monchai |
| author_facet | Labs, KBTG : Lertpiya, Anuruth Khamnuansin, Danupat Sucharitpongpan, Kantapong Balee, Pornchanan Chalothorn, Tawunrat Pongthawornkamol, Thadpong Lertsutthiwong, Monchai |
| contents | Large Language Models (LLMs) have demonstrated significant potential across various domains, particularly in banking and finance, where they can automate complex tasks and enhance decision-making at scale. Due to privacy, security, and regulatory concerns, organizations often prefer on-premise deployment of LLMs. The ThaiLLM initiative aims to enhance Thai language capabilities in open-LLMs, enabling Thai industry to leverage advanced language models. However, organizations often face a trade-off between deploying multiple specialized models versus the prohibitive expense of training a single multi-capability model. To address this, we explore model merging as a resource-efficient alternative for developing high-performance, multi-capability LLMs. We present results from two key experiments: first, merging Qwen-8B with ThaiLLM-8B demonstrates how ThaiLLM-8B enhances Thai general capabilities, showing an uplift of M3 and M6 O-NET exams over the general instruction-following Qwen-8B. Second, we merge Qwen-8B with both ThaiLLM-8B and THaLLE-CFA-8B. This combination results in further improvements in performance across both general and financial domains, by demonstrating an uplift in both M3 and M6 O-NET, Flare-CFA, and Thai-IC benchmarks. The report showcases the viability of model merging for efficiently creating multi-capability LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04597 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | THaLLE-ThaiLLM: Domain-Specialized Small LLMs for Finance and Thai -- Technical Report Labs, KBTG : Lertpiya, Anuruth Khamnuansin, Danupat Sucharitpongpan, Kantapong Balee, Pornchanan Chalothorn, Tawunrat Pongthawornkamol, Thadpong Lertsutthiwong, Monchai Computation and Language Large Language Models (LLMs) have demonstrated significant potential across various domains, particularly in banking and finance, where they can automate complex tasks and enhance decision-making at scale. Due to privacy, security, and regulatory concerns, organizations often prefer on-premise deployment of LLMs. The ThaiLLM initiative aims to enhance Thai language capabilities in open-LLMs, enabling Thai industry to leverage advanced language models. However, organizations often face a trade-off between deploying multiple specialized models versus the prohibitive expense of training a single multi-capability model. To address this, we explore model merging as a resource-efficient alternative for developing high-performance, multi-capability LLMs. We present results from two key experiments: first, merging Qwen-8B with ThaiLLM-8B demonstrates how ThaiLLM-8B enhances Thai general capabilities, showing an uplift of M3 and M6 O-NET exams over the general instruction-following Qwen-8B. Second, we merge Qwen-8B with both ThaiLLM-8B and THaLLE-CFA-8B. This combination results in further improvements in performance across both general and financial domains, by demonstrating an uplift in both M3 and M6 O-NET, Flare-CFA, and Thai-IC benchmarks. The report showcases the viability of model merging for efficiently creating multi-capability LLMs. |
| title | THaLLE-ThaiLLM: Domain-Specialized Small LLMs for Finance and Thai -- Technical Report |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.04597 |