On the Limitations of Steering in Language Model Alignment
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908346390413312 |
|---|---|
| author | Niranjan, Chebrolu Jaidka, Kokil Yeo, Gerard Christopher |
| author_facet | Niranjan, Chebrolu Jaidka, Kokil Yeo, Gerard Christopher |
| contents | Steering vectors are a promising approach to aligning language model behavior at inference time. In this paper, we propose a framework to assess the limitations of steering vectors as alignment mechanisms. Using a framework of transformer hook interventions and antonym-based function vectors, we evaluate the role of prompt structure and context complexity in steering effectiveness. Our findings indicate that steering vectors are promising for specific alignment tasks, such as value alignment, but may not provide a robust foundation for general-purpose alignment in LLMs, particularly in complex scenarios. We establish a methodological foundation for future investigations into steering capabilities of reasoning models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_01162 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Limitations of Steering in Language Model Alignment Niranjan, Chebrolu Jaidka, Kokil Yeo, Gerard Christopher Computation and Language Artificial Intelligence Steering vectors are a promising approach to aligning language model behavior at inference time. In this paper, we propose a framework to assess the limitations of steering vectors as alignment mechanisms. Using a framework of transformer hook interventions and antonym-based function vectors, we evaluate the role of prompt structure and context complexity in steering effectiveness. Our findings indicate that steering vectors are promising for specific alignment tasks, such as value alignment, but may not provide a robust foundation for general-purpose alignment in LLMs, particularly in complex scenarios. We establish a methodological foundation for future investigations into steering capabilities of reasoning models. |
| title | On the Limitations of Steering in Language Model Alignment |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2505.01162 |