Language Surgery in Multilingual Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914089076260864 |
|---|---|
| author | Lopo, Joanito Agili Habibi, Muhammad Ravi Shulthan Wong, Tack Hwa Ghozali, Muhammad Ilham Koto, Fajri Winata, Genta Indra Limkonchotiwat, Peerat Aji, Alham Fikri Cahyawijaya, Samuel |
| author_facet | Lopo, Joanito Agili Habibi, Muhammad Ravi Shulthan Wong, Tack Hwa Ghozali, Muhammad Ilham Koto, Fajri Winata, Genta Indra Limkonchotiwat, Peerat Aji, Alham Fikri Cahyawijaya, Samuel |
| contents | Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC's strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their monolingual and cross-lingual performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12450 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Language Surgery in Multilingual Large Language Models Lopo, Joanito Agili Habibi, Muhammad Ravi Shulthan Wong, Tack Hwa Ghozali, Muhammad Ilham Koto, Fajri Winata, Genta Indra Limkonchotiwat, Peerat Aji, Alham Fikri Cahyawijaya, Samuel Computation and Language Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC's strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their monolingual and cross-lingual performance. |
| title | Language Surgery in Multilingual Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.12450 |