When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911311961522176 |
|---|---|
| author | Zhao, Weixiang Guo, Jiahe Deng, Yang Wu, Tongtong Zhang, Wenxuan Hu, Yulin Sui, Xingyu Zhao, Yanyan Che, Wanxiang Qin, Bing Chua, Tat-Seng Liu, Ting |
| author_facet | Zhao, Weixiang Guo, Jiahe Deng, Yang Wu, Tongtong Zhang, Wenxuan Hu, Yulin Sui, Xingyu Zhao, Yanyan Che, Wanxiang Qin, Bing Chua, Tat-Seng Liu, Ting |
| contents | Multilingual reasoning remains a significant challenge for large language models (LLMs), with performance disproportionately favoring high-resource languages. Drawing inspiration from cognitive neuroscience, which suggests that human reasoning functions largely independently of language processing, we hypothesize that LLMs similarly encode reasoning and language as separable components that can be disentangled to enhance multilingual reasoning. To evaluate this, we perform a causal intervention by ablating language-specific representations at inference time. Experiments on 10 open-weight LLMs spanning 11 typologically diverse languages show that this language-specific ablation consistently boosts multilingual reasoning performance. Layer-wise analyses further confirm that language and reasoning representations can be effectively disentangled throughout the model, yielding improved multilingual reasoning capabilities, while preserving top-layer language features remains essential for maintaining linguistic fidelity. Compared to post-training methods such as supervised fine-tuning or reinforcement learning, our training-free language-reasoning disentanglement achieves comparable or superior results with minimal computational overhead. These findings shed light on the internal mechanisms underlying multilingual reasoning in LLMs and suggest a lightweight and interpretable strategy for improving cross-lingual generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15257 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners Zhao, Weixiang Guo, Jiahe Deng, Yang Wu, Tongtong Zhang, Wenxuan Hu, Yulin Sui, Xingyu Zhao, Yanyan Che, Wanxiang Qin, Bing Chua, Tat-Seng Liu, Ting Computation and Language Multilingual reasoning remains a significant challenge for large language models (LLMs), with performance disproportionately favoring high-resource languages. Drawing inspiration from cognitive neuroscience, which suggests that human reasoning functions largely independently of language processing, we hypothesize that LLMs similarly encode reasoning and language as separable components that can be disentangled to enhance multilingual reasoning. To evaluate this, we perform a causal intervention by ablating language-specific representations at inference time. Experiments on 10 open-weight LLMs spanning 11 typologically diverse languages show that this language-specific ablation consistently boosts multilingual reasoning performance. Layer-wise analyses further confirm that language and reasoning representations can be effectively disentangled throughout the model, yielding improved multilingual reasoning capabilities, while preserving top-layer language features remains essential for maintaining linguistic fidelity. Compared to post-training methods such as supervised fine-tuning or reinforcement learning, our training-free language-reasoning disentanglement achieves comparable or superior results with minimal computational overhead. These findings shed light on the internal mechanisms underlying multilingual reasoning in LLMs and suggest a lightweight and interpretable strategy for improving cross-lingual generalization. |
| title | When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.15257 |