Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908818459328512 |
|---|---|
| author | Han, Pengrui Xu, Xueqiang Xuan, Keyang Song, Peiyang Ouyang, Siru Tian, Runchu Jiang, Yuqing Qian, Cheng Jiang, Pengcheng Sun, Jiashuo Cui, Junxia Zhong, Ming Liu, Ge Han, Jiawei You, Jiaxuan |
| author_facet | Han, Pengrui Xu, Xueqiang Xuan, Keyang Song, Peiyang Ouyang, Siru Tian, Runchu Jiang, Yuqing Qian, Cheng Jiang, Pengcheng Sun, Jiashuo Cui, Junxia Zhong, Ming Liu, Ge Han, Jiawei You, Jiaxuan |
| contents | Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07276 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs Han, Pengrui Xu, Xueqiang Xuan, Keyang Song, Peiyang Ouyang, Siru Tian, Runchu Jiang, Yuqing Qian, Cheng Jiang, Pengcheng Sun, Jiashuo Cui, Junxia Zhong, Ming Liu, Ge Han, Jiawei You, Jiaxuan Artificial Intelligence Computation and Language Machine Learning Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs. |
| title | Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.07276 |