Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.05057 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912415505973248 |
|---|---|
| author | Ofer, Moshe Zamler, Orel Azaria, Amos |
| author_facet | Ofer, Moshe Zamler, Orel Azaria, Amos |
| contents | Large Language Models (LLMs) excel in high-resource languages but struggle with low-resource languages due to limited training data. This paper presents TALL (Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages), which integrates an LLM with two bilingual translation models. TALL transforms low-resource inputs into high-resource representations, leveraging the LLM's capabilities while preserving linguistic features through dimension alignment layers and custom transformers. Our experiments on Hebrew demonstrate significant improvements over several baselines, including direct use, naive translation, and fine-tuning approaches. The architecture employs a parameter-efficient strategy, freezing pre-trained components while training only lightweight adapter modules, balancing computational efficiency with performance gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_05057 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TALL -- A Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages Ofer, Moshe Zamler, Orel Azaria, Amos Computation and Language Artificial Intelligence Large Language Models (LLMs) excel in high-resource languages but struggle with low-resource languages due to limited training data. This paper presents TALL (Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages), which integrates an LLM with two bilingual translation models. TALL transforms low-resource inputs into high-resource representations, leveraging the LLM's capabilities while preserving linguistic features through dimension alignment layers and custom transformers. Our experiments on Hebrew demonstrate significant improvements over several baselines, including direct use, naive translation, and fine-tuning approaches. The architecture employs a parameter-efficient strategy, freezing pre-trained components while training only lightweight adapter modules, balancing computational efficiency with performance gains. |
| title | TALL -- A Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.05057 |