Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908792213471232 |
|---|---|
| author | Dong, Fangan Yan, Zuming Ge, Xuri Xu, Zhiwei Zhang, Mengqi Chen, Xuanang He, Ben Xin, Xin Chen, Zhumin Zhou, Ying |
| author_facet | Dong, Fangan Yan, Zuming Ge, Xuri Xu, Zhiwei Zhang, Mengqi Chen, Xuanang He, Ben Xin, Xin Chen, Zhumin Zhou, Ying |
| contents | Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationally expensive sampling strategies, limiting their practical efficiency. In this work, we first show that a small subset of neurons in LLMs exhibits strong predictive correlations with reasoning correctness. Based on this observation, we propose AdaRAS (Adaptive Reasoning Activation Steering), a lightweight test-time framework that improves reasoning reliability by selectively intervening on neuron activations. AdaRAS identifies Reasoning-Critical Neurons (RCNs) via a polarity-aware mean-difference criterion and adaptively steers their activations during inference, enhancing incorrect reasoning traces while avoiding degradation on already-correct cases. Experiments on 10 mathematics and coding benchmarks demonstrate consistent improvements, including over 13% gains on AIME-24 and AIME-25. Moreover, AdaRAS exhibits strong transferability across datasets and scalability to stronger models, outperforming post-training methods without additional training or sampling cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19847 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering Dong, Fangan Yan, Zuming Ge, Xuri Xu, Zhiwei Zhang, Mengqi Chen, Xuanang He, Ben Xin, Xin Chen, Zhumin Zhou, Ying Computation and Language Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationally expensive sampling strategies, limiting their practical efficiency. In this work, we first show that a small subset of neurons in LLMs exhibits strong predictive correlations with reasoning correctness. Based on this observation, we propose AdaRAS (Adaptive Reasoning Activation Steering), a lightweight test-time framework that improves reasoning reliability by selectively intervening on neuron activations. AdaRAS identifies Reasoning-Critical Neurons (RCNs) via a polarity-aware mean-difference criterion and adaptively steers their activations during inference, enhancing incorrect reasoning traces while avoiding degradation on already-correct cases. Experiments on 10 mathematics and coding benchmarks demonstrate consistent improvements, including over 13% gains on AIME-24 and AIME-25. Moreover, AdaRAS exhibits strong transferability across datasets and scalability to stronger models, outperforming post-training methods without additional training or sampling cost. |
| title | Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steering |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.19847 |