Cover Image

CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ye, Xinge, Wang, Rui, Wu, Yuchuan, Ma, Victor, Fang, Feiteng, Huang, Fei, Li, Yongbin
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity
by: Fang, Feiteng, et al.
Published: (2025)

Reverse Preference Optimization for Complex Instruction Following
by: Huang, Xiang, et al.
Published: (2025)

EPO: Explicit Policy Optimization for Strategic Reasoning in LLMs via Reinforcement Learning
by: Liu, Xiaoqian, et al.
Published: (2025)

Agentic Reinforcement Learning with Implicit Step Rewards
by: Liu, Xiaoqian, et al.
Published: (2025)

Fine-Tuning Language Models with Reward Learning on Policy
by: Lang, Hao, et al.
Published: (2024)

MOA: Multi-Objective Alignment for Role-Playing Agents
by: Liao, Chonghua, et al.
Published: (2025)

SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents
by: Si, Shuzheng, et al.
Published: (2023)

Reward Modeling from Natural Language Human Feedback
by: Wang, Zongqi, et al.
Published: (2026)

OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction
by: Zhang, Haonan, et al.
Published: (2025)

P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling
by: Zhang, Pinyi, et al.
Published: (2026)

Improving Factual Consistency of News Summarization by Contrastive Preference Optimization
by: Feng, Huawen, et al.
Published: (2023)

SDPO: Segment-Level Direct Preference Optimization for Social Agents
by: Kong, Aobo, et al.
Published: (2025)

FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents
by: Xiao, Ruixuan, et al.
Published: (2024)

Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts
by: Lang, Hao, et al.
Published: (2023)

Understanding Generalization in Role-Playing Models via Information Theory
by: Li, Yongqi, et al.
Published: (2025)

Adaptive Social Learning via Mode Policy Optimization for Language Agents
by: Wang, Minzheng, et al.
Published: (2025)

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
by: Chen, Yuhan, et al.
Published: (2023)

CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
by: Tang, Yihong, et al.
Published: (2026)

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization
by: Zhang, Xinghua, et al.
Published: (2024)

RoleRMBench & RoleRM: Towards Reward Modeling for Profile-Based Role Play in Dialogue Systems
by: Ding, Hang, et al.
Published: (2025)

Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards
by: Zeng, Xia, et al.
Published: (2025)

AmbigNLG: Addressing Task Ambiguity in Instruction for NLG
by: Niwa, Ayana, et al.
Published: (2024)

MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialogues
by: Chen, Weishu, et al.
Published: (2025)

Debate Helps Weak-to-Strong Generalization
by: Lang, Hao, et al.
Published: (2025)

Selective Weak-to-Strong Generalization
by: Lang, Hao, et al.
Published: (2025)

Socratic-PRMBench: Benchmarking Process Reward Models with Systematic Reasoning Patterns
by: Li, Xiang, et al.
Published: (2025)

DEMO: Reframing Dialogue Interaction with Fine-grained Element Modeling
by: Wang, Minzheng, et al.
Published: (2024)

Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration
by: Ma, Yingwei, et al.
Published: (2024)

Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
by: Zhou, Qinhao, et al.
Published: (2024)

Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing
by: Peng, Letian, et al.
Published: (2024)

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
by: Yu, Le, et al.
Published: (2024)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
by: Bai, Yang, et al.
Published: (2026)

A Survey on Self-Evolution of Large Language Models
by: Tao, Zhengwei, et al.
Published: (2024)

On-Policy RL with Optimal Reward Baseline
by: Hao, Yaru, et al.
Published: (2025)

MC-CPO: Mastery-Conditioned Constrained Policy Optimization
by: Olukola, Oluseyi, et al.
Published: (2026)

TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligence
by: Hou, Guiyang, et al.
Published: (2025)

ReDit: Reward Dithering for Improved LLM Policy Optimization
by: Wei, Chenxing, et al.
Published: (2025)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

BOOKMARKS: Efficient Active Storyline Memory for Role-playing
by: Peng, Letian, et al.
Published: (2026)

Reward Difference Optimization For Sample Reweighting In Offline RLHF
by: Wang, Shiqi, et al.
Published: (2024)