Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909901065814016 |
|---|---|
| author | Horoi, Stefan Cho, Sangwoo Chakraborty, Supriyo Zhang, Shi-Xiong Sahu, Sambit Wolf, Guy Winata, Genta Indra |
| author_facet | Horoi, Stefan Cho, Sangwoo Chakraborty, Supriyo Zhang, Shi-Xiong Sahu, Sambit Wolf, Guy Winata, Genta Indra |
| contents | Task arithmetic is a powerful technique for transferring skills between Large Language Models (LLMs), but it often suffers from negative interference when models have diverged during training. We address this limitation by first aligning the models' parameter spaces, leveraging the inherent permutation, rotation, and scaling symmetries of Transformer architectures. We adapt parameter space alignment for modern Grouped-Query Attention (GQA) and SwiGLU layers, exploring both weight-based and activation-based approaches. Using this alignment-first strategy, we successfully transfer advanced reasoning skills to a non-reasoning model. Experiments on challenging reasoning benchmarks show that our method consistently outperforms standard task arithmetic. This work provides an effective approach for merging and transferring specialized skills across evolving LLM families, reducing redundant fine-tuning and enhancing model adaptability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10850 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs Horoi, Stefan Cho, Sangwoo Chakraborty, Supriyo Zhang, Shi-Xiong Sahu, Sambit Wolf, Guy Winata, Genta Indra Computation and Language Artificial Intelligence Machine Learning Task arithmetic is a powerful technique for transferring skills between Large Language Models (LLMs), but it often suffers from negative interference when models have diverged during training. We address this limitation by first aligning the models' parameter spaces, leveraging the inherent permutation, rotation, and scaling symmetries of Transformer architectures. We adapt parameter space alignment for modern Grouped-Query Attention (GQA) and SwiGLU layers, exploring both weight-based and activation-based approaches. Using this alignment-first strategy, we successfully transfer advanced reasoning skills to a non-reasoning model. Experiments on challenging reasoning benchmarks show that our method consistently outperforms standard task arithmetic. This work provides an effective approach for merging and transferring specialized skills across evolving LLM families, reducing redundant fine-tuning and enhancing model adaptability. |
| title | Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2511.10850 |