Cover Image

R$^2$PO: Decoupling Training Trajectories from Inference Responses for LLM Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jingchu, Xu, Bingbing, Yuan, Yige, Xie, Bin, Sun, Xiaoqian, Shen, Huawei
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multi-Personality Generation of LLMs at Decoding-time
by: Chen, Rongxin, et al.
Published: (2025)

Inference-time Alignment in Continuous Space
by: Yuan, Yige, et al.
Published: (2025)

Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning
by: Wang, Zixu, et al.
Published: (2024)

Fact-Level Confidence Calibration and Self-Correction
by: Yuan, Yige, et al.
Published: (2024)

From Outcomes to Processes: Guiding PRM Learning from ORM for Inference-Time Alignment
by: Xie, Bin, et al.
Published: (2025)

Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning
by: Yuan, Yurun, et al.
Published: (2025)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
by: Zhao, Anhao, et al.
Published: (2026)

Incentivizing Strong Reasoning from Weak Supervision
by: Yuan, Yige, et al.
Published: (2025)

Reinforce LLM Reasoning through Multi-Agent Reflection
by: Yuan, Yurun, et al.
Published: (2025)

Where Did It Go Wrong? Attributing Undesirable LLM Behaviors via Representation Gradient Tracing
by: Li, Zhe, et al.
Published: (2025)

Momentum Streams for Optimizer-Inspired Transformers
by: Gai, Jingchu, et al.
Published: (2026)

LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals
by: Sun, Lihao, et al.
Published: (2026)

Breaking the Capability Ceiling of LLM Post-Training by Reintroducing Markov States
by: Yuan, Yurun, et al.
Published: (2026)

R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning
by: Chen, Zhuokun, et al.
Published: (2025)

V-STaR: Training Verifiers for Self-Taught Reasoners
by: Hosseini, Arian, et al.
Published: (2024)

Transparent Screening for LLM Inference and Training Impacts
by: Pachot, Arnault, et al.
Published: (2026)

On Designing Effective RL Reward at Training Time for LLM Reasoning
by: Gao, Jiaxuan, et al.
Published: (2024)

Diagnosing Training Inference Mismatch in LLM Reinforcement Learning
by: Zhong, Tianle, et al.
Published: (2026)

Training-Trajectory-Aware Token Selection
by: Shen, Zhanming, et al.
Published: (2026)

SABER: Switchable and Balanced Training for Efficient LLM Reasoning
by: Zhao, Kai, et al.
Published: (2025)

ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
by: Yang, Ling, et al.
Published: (2025)

Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Generation for Multi-Agent Systems
by: Wang, Zixu, et al.
Published: (2026)

Advancing LLM Reasoning Generalists with Preference Trees
by: Yuan, Lifan, et al.
Published: (2024)

PITA: Preference-Guided Inference-Time Alignment for LLM Post-Training
by: Bobbili, Sarat Chandra, et al.
Published: (2025)

Beyond Entangled Planning: Task-Decoupled Planning for Long-Horizon Agents
by: Li, Yunfan, et al.
Published: (2026)

Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
by: Ma, Zhengzhao, et al.
Published: (2026)

The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis
by: Wei, Zihao, et al.
Published: (2025)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
by: Pan, Bowen, et al.
Published: (2024)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
by: Hu, Xing, et al.
Published: (2024)

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
by: Liu, Yixin, et al.
Published: (2026)

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs
by: Kang, Feiyang, et al.
Published: (2024)

SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation
by: Xu, Ting, et al.
Published: (2025)

From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph
by: Gong, Junfeng, et al.
Published: (2025)

R-Zero: Self-Evolving Reasoning LLM from Zero Data
by: Huang, Chengsong, et al.
Published: (2025)

AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners
by: Koh, Woosung, et al.
Published: (2025)

Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning
by: Zhou, Cai, et al.
Published: (2026)

PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoning
by: Lu, Wenquan, et al.
Published: (2026)

Scaling with Collapse: Efficient and Predictable Training of LLM Families
by: Bergsma, Shane, et al.
Published: (2025)

On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
by: Zhang, Yifan, et al.
Published: (2025)

Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation
by: Bsharat, Sondos Mahmoud, et al.
Published: (2025)