Contextual Linear Activation Steering of Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917440447840256 |
|---|---|
| author | Hsu, Brandon Beaglehole, Daniel Radhakrishnan, Adityanarayanan Belkin, Mikhail |
| author_facet | Hsu, Brandon Beaglehole, Daniel Radhakrishnan, Adityanarayanan Belkin, Mikhail |
| contents | Linear activation steering is a powerful approach for eliciting the capabilities of large language models and specializing their behavior using limited labeled data. While effective, existing methods often apply a fixed steering strength to all tokens, resulting in inconsistent steering quality across diverse input prompts. In this work, we introduce Contextual Linear Activation Steering (CLAS), a method that dynamically adapts linear activation steering to context-dependent steering strengths. Across eleven steering benchmarks and four model families, it consistently outperforms standard linear activation steering and matches or exceeds the performance of ReFT and LoRA in settings with limited labeled data. We therefore propose CLAS as a scalable, interpretable, and accurate method for specializing and steering large language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_24693 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Contextual Linear Activation Steering of Language Models Hsu, Brandon Beaglehole, Daniel Radhakrishnan, Adityanarayanan Belkin, Mikhail Computation and Language Linear activation steering is a powerful approach for eliciting the capabilities of large language models and specializing their behavior using limited labeled data. While effective, existing methods often apply a fixed steering strength to all tokens, resulting in inconsistent steering quality across diverse input prompts. In this work, we introduce Contextual Linear Activation Steering (CLAS), a method that dynamically adapts linear activation steering to context-dependent steering strengths. Across eleven steering benchmarks and four model families, it consistently outperforms standard linear activation steering and matches or exceeds the performance of ReFT and LoRA in settings with limited labeled data. We therefore propose CLAS as a scalable, interpretable, and accurate method for specializing and steering large language models. |
| title | Contextual Linear Activation Steering of Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.24693 |