Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916812113838080 |
|---|---|
| author | Kim, Jinpyo Cho, Gyeongje Park, Chanwoo Park, Jongwon Kim, Jongmin So, Yeonkyoun Lee, Jaejin |
| author_facet | Kim, Jinpyo Cho, Gyeongje Park, Chanwoo Park, Jongwon Kim, Jongmin So, Yeonkyoun Lee, Jaejin |
| contents | Since state-of-the-art LLMs often underperform in languages other than English or Chinese, improving the capability of LLMs in new languages has become an essential task. Moreover, LLMs' entire end-to-end training process remains largely unknown to the public due to proprietary reasons, technical complexity, inconsistent documentation, and ethical considerations. The complete picture remains a closely guarded secret within the industry. This paper presents methods to adapt an existing English-based LLM to Korean in a low-budget scenario. We describe the entire end-to-end process: collecting Korean datasets, preprocessing the data, training the model, creating downstream benchmarks, and conducting evaluations. The evaluation results indicate that our method can effectively and cost-efficiently add new language capabilities to existing LLMs. Our new bilingual models, Thunder-LLM and Thunder-LLM-Ins, achieve superior Korean performance compared to state-of-the-art models while utilizing minimal data and computational resources. We share our comprehensive experience and make the code publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_21595 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources Kim, Jinpyo Cho, Gyeongje Park, Chanwoo Park, Jongwon Kim, Jongmin So, Yeonkyoun Lee, Jaejin Computation and Language Since state-of-the-art LLMs often underperform in languages other than English or Chinese, improving the capability of LLMs in new languages has become an essential task. Moreover, LLMs' entire end-to-end training process remains largely unknown to the public due to proprietary reasons, technical complexity, inconsistent documentation, and ethical considerations. The complete picture remains a closely guarded secret within the industry. This paper presents methods to adapt an existing English-based LLM to Korean in a low-budget scenario. We describe the entire end-to-end process: collecting Korean datasets, preprocessing the data, training the model, creating downstream benchmarks, and conducting evaluations. The evaluation results indicate that our method can effectively and cost-efficiently add new language capabilities to existing LLMs. Our new bilingual models, Thunder-LLM and Thunder-LLM-Ins, achieve superior Korean performance compared to state-of-the-art models while utilizing minimal data and computational resources. We share our comprehensive experience and make the code publicly available. |
| title | Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.21595 |