The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913091693838336 |
|---|---|
| author | Balasubramanian, Rishab Lin, Pin-Jie Sharma, Rituraj Fang, Anjie Abdi, Fardin Rozgic, Viktor Du, Zheng Bansal, Mohit Vu, Tu |
| author_facet | Balasubramanian, Rishab Lin, Pin-Jie Sharma, Rituraj Fang, Anjie Abdi, Fardin Rozgic, Viktor Du, Zheng Bansal, Mohit Vu, Tu |
| contents | We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_06377 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment Balasubramanian, Rishab Lin, Pin-Jie Sharma, Rituraj Fang, Anjie Abdi, Fardin Rozgic, Viktor Du, Zheng Bansal, Mohit Vu, Tu Machine Learning Artificial Intelligence We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories. |
| title | The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2604.06377 |