Cover Image

Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Qingru, Qiu, Liang, Hong, Ilgee, Xu, Zhenghao, Liu, Tianyi, Li, Shiyang, Zhang, Rongzhi, Li, Zheng, Li, Lihong, Yin, Bing, Zhang, Chao, Chen, Jianshu, Jiang, Haoming, Zhao, Tuo
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Ask a Strong LLM Judge when Your Reward Model is Uncertain
by: Xu, Zhenghao, et al.
Published: (2025)

Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
by: Hong, Ilgee, et al.
Published: (2025)

Robust Reinforcement Learning from Corrupted Human Feedback
by: Bukharin, Alexander, et al.
Published: (2024)

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
by: Guo, Siqi, et al.
Published: (2025)

RNR: Teaching Large Language Models to Follow Roles and Rules
by: Wang, Kuan, et al.
Published: (2024)

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback
by: Hong, Ilgee, et al.
Published: (2024)

Data Diversity Matters for Robust Instruction Tuning
by: Bukharin, Alexander, et al.
Published: (2023)

SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
by: Li, Zichong, et al.
Published: (2025)

Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
by: Xu, Siyuan, et al.
Published: (2026)

BackPlay: Head-Only Look-Back Self-Correction for Diffusion Language Models
by: Liu, Liming, et al.
Published: (2026)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
by: Heng, Yuzhao, et al.
Published: (2024)

Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs
by: Li, Junbo, et al.
Published: (2025)

OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
by: Liu, Tianci, et al.
Published: (2025)

MEMORYLLM: Towards Self-Updatable Large Language Models
by: Wang, Yu, et al.
Published: (2024)

Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards
by: Jiayang, Cheng, et al.
Published: (2026)

Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations
by: Wei, Zeming, et al.
Published: (2023)

Sampling Complexity of TD and PPO in RKHS
by: Zou, Lu, et al.
Published: (2025)

A Study on Video Conference Fatigue: Moderating Effect of Mindfulness and Mediating Effect of Appearance Dissatisfaction
by: Bilei Zhou, et al.
Published: (2025)

Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
by: Zhang, Yuwei, et al.
Published: (2026)

Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLP
by: Zhang, Lihong, et al.
Published: (2025)

Attention-Aligned Reasoning for Large Language Models
by: Zhang, Hongxiang, et al.
Published: (2025)

Aligning Large Language Models with Representation Editing: A Control Perspective
by: Kong, Lingkai, et al.
Published: (2024)

VRank: Enhancing Verilog Code Generation from Large Language Models via Self-Consistency
by: Zhao, Zhuorui, et al.
Published: (2025)

14‐1: Exploration of 40Hz Infrared Rhythmic Light Stimulation in Healthy Display Applications
by: Shiyang Song, et al.
Published: (2025)

Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards
by: Li, Ming, et al.
Published: (2025)

Lightweight Backbone Networks Only Require Adaptive Lightweight Self-Attention Mechanisms
by: Li, Fengyun, et al.
Published: (2025)

MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework based on Large language Model
by: Zhang, Chengze, et al.
Published: (2025)

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
by: Zhuang, Yuchen, et al.
Published: (2025)

Investigation of Plasma Mixing Processes in the Context of Indirect Drive Inertial Confinement Fusion
by: Li, Xiaoran, et al.
Published: (2025)

Continual Learning Using Only Large Language Model Prompting
by: Qiu, Jiabao, et al.
Published: (2024)

SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks
by: Wang, Tianyi, et al.
Published: (2026)

Considerations of underground factory excavation and temperature stress on rock bolt crane beam
by: Haoming Zhang, et al.
Published: (2025)

Non-Asymptotic Global Convergence of PPO-Clip
by: Liu, Yin, et al.
Published: (2025)

Style Adaptation for Domain-adaptive Semantic Segmentation
by: Li, Ting, et al.
Published: (2024)

NeuronMoE: Neuron-Guided Mixture-of-Experts for Efficient Multilingual LLM Extension
by: Li, Rongzhi, et al.
Published: (2026)

IHEval: Evaluating Language Models on Following the Instruction Hierarchy
by: Zhang, Zhihan, et al.
Published: (2025)

SpeechAlign: Aligning Speech Generation to Human Preferences
by: Zhang, Dong, et al.
Published: (2024)

Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training
by: Xu, Ran, et al.
Published: (2026)

Selective Attention Federated Learning: Improving Privacy and Efficiency for Clinical Text Classification
by: Li, Yue, et al.
Published: (2025)