Cover Image

PSPO*: An Effective Process-supervised Policy Optimization for Reasoning Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Jiawei, Liang, Xinyue, Zhang, Junlong, Yang, Yizhe, Feng, Chong, Gao, Yang
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Alignment: Expanding Reasoning Capacity via Manifold-Reshaping Policy Optimization
by: Wang, Dayu, et al.
Published: (2026)

GRPO-VPS: Enhancing Group Relative Policy Optimization with Verifiable Process Supervision for Effective Reasoning
by: Wang, Jingyi, et al.
Published: (2026)

An Optimization Algorithm for Multimodal Data Alignment
by: Zhang, Wei, et al.
Published: (2025)

Unveiling and Addressing Pseudo Forgetting in Large Language Models
by: Sun, Huashan, et al.
Published: (2024)

PolicyLong: Towards On-Policy Context Extension
by: Jia, Junlong, et al.
Published: (2026)

DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
by: Li, Gang, et al.
Published: (2025)

V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization
by: Jiang, Yubo, et al.
Published: (2026)

ERPO: Token-Level Entropy-Regulated Policy Optimization for Large Reasoning Models
by: Yu, Song, et al.
Published: (2026)

Segment-Aligned Policy Optimization for Multi-Modal Reasoning
by: Gao, Lei, et al.
Published: (2026)

Policy Guided Tree Search for Enhanced LLM Reasoning
by: Li, Yang
Published: (2025)

The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
by: Agarwal, Shivam, et al.
Published: (2025)

Relative Policy-Transition Optimization for Fast Policy Transfer
by: Xu, Jiawei, et al.
Published: (2022)

EP-GRPO: Entropy-Progress Aligned Group Relative Policy Optimization with Implicit Process Guidance
by: Yu, Song, et al.
Published: (2026)

Flow-based Policy With Distributional Reinforcement Learning in Trajectory Optimization
by: Hao, Ruijie, et al.
Published: (2026)

Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning
by: Li, Xuan, et al.
Published: (2026)

PVPO: Pre-Estimated Value-Based Policy Optimization for Agentic Reasoning
by: Feng, Wenfeng, et al.
Published: (2025)

Calibration-Aware Policy Optimization for Reasoning LLMs
by: Wang, Ziqi, et al.
Published: (2026)

Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning
by: Yang, Zhaohui, et al.
Published: (2025)

Towards Flash Thinking via Decoupled Advantage Policy Optimization
by: Tan, Zezhong, et al.
Published: (2025)

UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning
by: Zhang, Jiawei, et al.
Published: (2025)

Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning
by: Gao, Chen-Xiao, et al.
Published: (2025)

TreeRPO: Tree Relative Policy Optimization
by: Yang, Zhicheng, et al.
Published: (2025)

Not All Data are Good Labels: On the Self-supervised Labeling for Time Series Forecasting
by: Yang, Yuxuan, et al.
Published: (2025)

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
by: Zhang, Yulin, et al.
Published: (2026)

Beyond Importance Sampling: Rejection-Gated Policy Optimization
by: Sun, Ziwu, et al.
Published: (2026)

Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps
by: Yang, Ningyuan, et al.
Published: (2025)

Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflection
by: Zhao, Zihui, et al.
Published: (2025)

Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth
by: Zhang, Jiawei, et al.
Published: (2025)

Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards
by: Fan, Jiajun, et al.
Published: (2025)

Hybrid Reward Normalization for Process-supervised Non-verifiable Agentic Tasks
by: Xu, Peiran, et al.
Published: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

LEPO: Latent Reasoning Policy Optimization for Large Language Models
by: Zhou, Yuyan, et al.
Published: (2026)

HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
by: Zeng, Xinyue, et al.
Published: (2026)

A Variance-Reduced Cubic-Regularized Newton for Policy Optimization
by: Sun, Cheng, et al.
Published: (2025)

OGLS-SD: On-Policy Self-Distillation with Outcome-Guided Logit Steering for LLM Reasoning
by: Yang, Yuxiao, et al.
Published: (2026)

Breaking $\textit{Winner-Takes-All}$: Cooperative Policy Optimization Improves Diverse LLM Reasoning
by: Chen, Haoxuan, et al.
Published: (2026)

Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes
by: Kobalczyk, Katarzyna, et al.
Published: (2024)

When Maximum Entropy Misleads Policy Optimization
by: Zhang, Ruipeng, et al.
Published: (2025)

SignalLLM: A General-Purpose LLM Agent Framework for Automated Signal Processing
by: Ke, Junlong, et al.
Published: (2025)

Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning
by: Yang, Zhicheng, et al.
Published: (2026)