Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916039495778304 |
|---|---|
| author | Yang, Junyao Qian, Chen Wang, Kun Zhang, Linfeng Zhang, Quanshi Liu, Yong Liu, Dongrui |
| author_facet | Yang, Junyao Qian, Chen Wang, Kun Zhang, Linfeng Zhang, Quanshi Liu, Yong Liu, Dongrui |
| contents | The advancement of Large Reasoning Models (LRMs) has catalyzed a paradigm shift from reactive ``fast thinking'' text generation to systematic, step-by-step ``slow thinking'' reasoning, unlocking state-of-the-art performance in complex mathematical and logical tasks. However, the field faces \textit{the fundamental gap between token-level behavioral analysis and internal reasoning mechanisms, and the instability of reinforcement learning (RL) for reasoning optimization relying on costly external verifiers}. We identify and formally define \textbf{Entropy-Gradient Inversion}, a robust negative correlation between token entropy and logit gradients that acts as a definitive geometric fingerprint for LRM reasoning capability. Building on this, we propose \textbf{Correlation-Regularized Group Policy Optimization (CorR-PO)}, which embeds this inversion signature into RL reward regularization. Extensive experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines, confirming that stronger inversion directly correlates with superior reasoning performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_17770 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models Yang, Junyao Qian, Chen Wang, Kun Zhang, Linfeng Zhang, Quanshi Liu, Yong Liu, Dongrui Artificial Intelligence Computation and Language The advancement of Large Reasoning Models (LRMs) has catalyzed a paradigm shift from reactive ``fast thinking'' text generation to systematic, step-by-step ``slow thinking'' reasoning, unlocking state-of-the-art performance in complex mathematical and logical tasks. However, the field faces \textit{the fundamental gap between token-level behavioral analysis and internal reasoning mechanisms, and the instability of reinforcement learning (RL) for reasoning optimization relying on costly external verifiers}. We identify and formally define \textbf{Entropy-Gradient Inversion}, a robust negative correlation between token entropy and logit gradients that acts as a definitive geometric fingerprint for LRM reasoning capability. Building on this, we propose \textbf{Correlation-Regularized Group Policy Optimization (CorR-PO)}, which embeds this inversion signature into RL reward regularization. Extensive experiments on various reasoning benchmarks across multiple model scales show CorR-PO consistently outperforms state-of-the-art baselines, confirming that stronger inversion directly correlates with superior reasoning performance. |
| title | Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2605.17770 |