CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911715365486592 |
|---|---|
| author | Tang, Yihong Chen, Kehai Yue, Liang Wang, Benyou Zhang, Min |
| author_facet | Tang, Yihong Chen, Kehai Yue, Liang Wang, Benyou Zhang, Min |
| contents | Recent advancements in Reinforcement Learning (RL), particularly Group Relative Policy Optimization (GRPO), have significantly enhanced the reasoning capabilities of Large Language Models. However, applying these problem-centric optimization methods to role-playing agents often leads to a loss of character fidelity and style collapse, as they prioritize context-specific utility over persona alignment. To address this, we propose Character-Centric Group Relative Policy Optimization (CRPO), a framework designed to realign RL objectives with the role-playing task. CRPO improves character distinctiveness through three mechanisms: decoupling task logic from stylistic rewards to resolve gradient conflicts, dynamically adapting optimization constraints based on character complexity, and utilizing generic responses as negative baselines to prevent the model from reverting to a common distribution. Extensive experiments demonstrate that CRPO outperforms existing methods in consistency, emotion and others. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_25511 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents Tang, Yihong Chen, Kehai Yue, Liang Wang, Benyou Zhang, Min Computation and Language Recent advancements in Reinforcement Learning (RL), particularly Group Relative Policy Optimization (GRPO), have significantly enhanced the reasoning capabilities of Large Language Models. However, applying these problem-centric optimization methods to role-playing agents often leads to a loss of character fidelity and style collapse, as they prioritize context-specific utility over persona alignment. To address this, we propose Character-Centric Group Relative Policy Optimization (CRPO), a framework designed to realign RL objectives with the role-playing task. CRPO improves character distinctiveness through three mechanisms: decoupling task logic from stylistic rewards to resolve gradient conflicts, dynamically adapting optimization constraints based on character complexity, and utilizing generic responses as negative baselines to prevent the model from reverting to a common distribution. Extensive experiments demonstrate that CRPO outperforms existing methods in consistency, emotion and others. |
| title | CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agents |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2605.25511 |