Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.09953 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910137619316736 |
|---|---|
| author | Nie, Shuaiyi Ding, Siyu Zhang, Wenyuan Yu, Linhao Yang, Tianmeng Chen, Yao Yin, Weichong Sun, Yu Wu, Hua Liu, Tingwen |
| author_facet | Nie, Shuaiyi Ding, Siyu Zhang, Wenyuan Yu, Linhao Yang, Tianmeng Chen, Yao Yin, Weichong Sun, Yu Wu, Hua Liu, Tingwen |
| contents | Large reasoning models trained with reinforcement learning and verifiable rewards (RLVR) achieve strong performance on complex reasoning tasks, yet often overthink, generating redundant reasoning without performance gains. Existing trajectory-level length penalties often fail to effectively shorten reasoning length and degrade accuracy, as they uniformly treat all reasoning steps and lack fine-grained signals to distinguish redundancy from necessity. Meanwhile, process-supervised methods are typically resource-intensive and suffer from inaccurate credit assignment. To address these issues, we propose ATTNPO, a low-overhead process-supervised RL framework that leverages the model's intrinsic attention signals for step-level credit assignment. We first identify a set of special attention heads that naturally focus on essential steps while suppressing redundant ones. By leveraging the attention scores of these heads, We then employ two sub-strategies to mitigate overthinking by discouraging redundant steps while preserving accuracy by reducing penalties on essential steps. Experimental results show that ATTNPO substantially reduces reasoning length while significantly improving performance across 9 benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_09953 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning Nie, Shuaiyi Ding, Siyu Zhang, Wenyuan Yu, Linhao Yang, Tianmeng Chen, Yao Yin, Weichong Sun, Yu Wu, Hua Liu, Tingwen Computation and Language Large reasoning models trained with reinforcement learning and verifiable rewards (RLVR) achieve strong performance on complex reasoning tasks, yet often overthink, generating redundant reasoning without performance gains. Existing trajectory-level length penalties often fail to effectively shorten reasoning length and degrade accuracy, as they uniformly treat all reasoning steps and lack fine-grained signals to distinguish redundancy from necessity. Meanwhile, process-supervised methods are typically resource-intensive and suffer from inaccurate credit assignment. To address these issues, we propose ATTNPO, a low-overhead process-supervised RL framework that leverages the model's intrinsic attention signals for step-level credit assignment. We first identify a set of special attention heads that naturally focus on essential steps while suppressing redundant ones. By leveraging the attention scores of these heads, We then employ two sub-strategies to mitigate overthinking by discouraging redundant steps while preserving accuracy by reducing penalties on essential steps. Experimental results show that ATTNPO substantially reduces reasoning length while significantly improving performance across 9 benchmarks. |
| title | ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.09953 |