Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916047494316032 |
|---|---|
| author | Yang, Xiao-Wen Han, Ziyu Zhang, Xi-Hua Wei, Wen-Da Shao, Jie-Jing Guo, Lan-Zhe Li, Yu-Feng |
| author_facet | Yang, Xiao-Wen Han, Ziyu Zhang, Xi-Hua Wei, Wen-Da Shao, Jie-Jing Guo, Lan-Zhe Li, Yu-Feng |
| contents | Looped Language Models (LoopLMs) enable efficient latent reasoning through depth recurrence, yet exhibit unreliable test-time scaling behavior: performance often peaks at a certain iteration depth and then collapses with further recurrence. Through latent dynamics analysis, we find an inherent trade-off between stability and effectiveness in existing architectures and strategies. By conceptualizing reasoning as uncertainty reduction, we propose that convergence toward stable fixed points while preserving effectiveness represents a promising way. To this end, we propose STARS (STAbility-driven Recurrent Scaling), a training framework that constrains latent states to approach asymptotically stable fixed points. This is realized via efficient Jacobian Spectral Radius Regularization with random loop sampling, enabling STARS to maximize effectiveness while ensuring rigorous stability. Experiments on arithmetic tasks show that STARS achieves reliable test-time scaling, and on complex mathematical reasoning it substantially mitigates performance degradation as recurrence depth increases while also improving peak performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_26733 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models Yang, Xiao-Wen Han, Ziyu Zhang, Xi-Hua Wei, Wen-Da Shao, Jie-Jing Guo, Lan-Zhe Li, Yu-Feng Machine Learning Artificial Intelligence Looped Language Models (LoopLMs) enable efficient latent reasoning through depth recurrence, yet exhibit unreliable test-time scaling behavior: performance often peaks at a certain iteration depth and then collapses with further recurrence. Through latent dynamics analysis, we find an inherent trade-off between stability and effectiveness in existing architectures and strategies. By conceptualizing reasoning as uncertainty reduction, we propose that convergence toward stable fixed points while preserving effectiveness represents a promising way. To this end, we propose STARS (STAbility-driven Recurrent Scaling), a training framework that constrains latent states to approach asymptotically stable fixed points. This is realized via efficient Jacobian Spectral Radius Regularization with random loop sampling, enabling STARS to maximize effectiveness while ensuring rigorous stability. Experiments on arithmetic tasks show that STARS achieves reliable test-time scaling, and on complex mathematical reasoning it substantially mitigates performance degradation as recurrence depth increases while also improving peak performance. |
| title | Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.26733 |