Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.21215 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915815259897856 |
|---|---|
| author | Hu, Runyi Zhang, Jie Zhao, Shiqian Meng, Jiale Li, Jiwei Zeng, Jason Wu, Ming Heinrich, Michael Wen, Yonggang Zhang, Tianwei |
| author_facet | Hu, Runyi Zhang, Jie Zhao, Shiqian Meng, Jiale Li, Jiwei Zeng, Jason Wu, Ming Heinrich, Michael Wen, Yonggang Zhang, Tianwei |
| contents | Token-level steering has emerged as a pivotal approach for inference-time alignment, enabling fine grained control over large language models by modulating their output distributions without parameter updates. While effective, existing methods rely on dense intervention at every decoding step. This persistent manipulation not only incurs substantial computational overhead but also risks compromising generation quality by excessively drifting from the model's intrinsic distribution. In this work, we show that dense intervention is unnecessary and propose Sparse Inference time Alignment (SIA), which performs sparse junction steering by intervening only at critical decision points along the generation trajectory. Our key insight is that high entropy junctions mark pivotal decision points in the generation trajectory and are particularly susceptible to misalignment, indicating the need to introduce alignment related reward signals at these points. Extensive experiments across different model families and alignment objectives show that steering only 20% to 80% of tokens achieves superior alignment-efficiency trade offs. For strong base models such as Qwen3, intervening on as few as 20% of tokens matches or even surpasses heavily post-trained instruct models. This sparsity enables stronger guidance while better preserving the model's native distribution, integrates seamlessly with search based methods such as Best-of-N, and reduces computational cost by up to 6x. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21215 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Inference-time Alignment via Sparse Junction Steering Hu, Runyi Zhang, Jie Zhao, Shiqian Meng, Jiale Li, Jiwei Zeng, Jason Wu, Ming Heinrich, Michael Wen, Yonggang Zhang, Tianwei Computation and Language Artificial Intelligence Token-level steering has emerged as a pivotal approach for inference-time alignment, enabling fine grained control over large language models by modulating their output distributions without parameter updates. While effective, existing methods rely on dense intervention at every decoding step. This persistent manipulation not only incurs substantial computational overhead but also risks compromising generation quality by excessively drifting from the model's intrinsic distribution. In this work, we show that dense intervention is unnecessary and propose Sparse Inference time Alignment (SIA), which performs sparse junction steering by intervening only at critical decision points along the generation trajectory. Our key insight is that high entropy junctions mark pivotal decision points in the generation trajectory and are particularly susceptible to misalignment, indicating the need to introduce alignment related reward signals at these points. Extensive experiments across different model families and alignment objectives show that steering only 20% to 80% of tokens achieves superior alignment-efficiency trade offs. For strong base models such as Qwen3, intervening on as few as 20% of tokens matches or even surpasses heavily post-trained instruct models. This sparsity enables stronger guidance while better preserving the model's native distribution, integrates seamlessly with search based methods such as Best-of-N, and reduces computational cost by up to 6x. |
| title | Inference-time Alignment via Sparse Junction Steering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.21215 |