Imagen de Portada

The Policy Cliff: A Theoretical Analysis of Reward-Policy Maps in Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Xu, Xingcheng
Formato:	Preprint
Publicado:	2025
Materias:	Artificial Intelligence Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Fine-Tuning Language Models with Reward Learning on Policy
por: Lang, Hao, et al.
Publicado: (2024)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
por: Zhao, Shiwan, et al.
Publicado: (2026)

Co-Evolution of Policy and Internal Reward for Language Agents
por: Wang, Xinyu, et al.
Publicado: (2026)

Large Language Models as Generalizable Policies for Embodied Tasks
por: Szot, Andrew, et al.
Publicado: (2023)

Aligning Large Language Models by On-Policy Self-Judgment
por: Lee, Sangkyu, et al.
Publicado: (2024)

It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
por: Lu, Taiming, et al.
Publicado: (2024)

Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
por: Chen, Yifang, et al.
Publicado: (2024)

Gradient-Adaptive Policy Optimization: Towards Multi-Objective Alignment of Large Language Models
por: Li, Chengao, et al.
Publicado: (2025)

Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Models
por: Guo, Yiran, et al.
Publicado: (2025)

Off-Policy Corrected Reward Modeling for Reinforcement Learning from Human Feedback
por: Ackermann, Johannes, et al.
Publicado: (2025)

Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
por: Li, Yaxuan, et al.
Publicado: (2026)

Leveraging Group Relative Policy Optimization to Advance Large Language Models in Traditional Chinese Medicine
por: Xie, Jiacheng, et al.
Publicado: (2025)

TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
por: Choo, Jinho, et al.
Publicado: (2026)

ReDit: Reward Dithering for Improved LLM Policy Optimization
por: Wei, Chenxing, et al.
Publicado: (2025)

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
por: Yang, Wenkai, et al.
Publicado: (2026)

Information-Theoretic Reward Decomposition for Generalizable RLHF
por: Mao, Liyuan, et al.
Publicado: (2025)

Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models
por: Lin, Nianyi, et al.
Publicado: (2025)

Policy Improvement using Language Feedback Models
por: Zhong, Victor, et al.
Publicado: (2024)

Using Large Language Models to Automate and Expedite Reinforcement Learning with Reward Machine
por: Alsadat, Shayan Meshkat, et al.
Publicado: (2024)

Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
por: Zhang, Qingru, et al.
Publicado: (2025)

Hindsight-Anchored Policy Optimization: Turning Failure into Feedback in Sparse Reward Settings
por: Wu, Yuning, et al.
Publicado: (2026)

A Comment On "The Illusion of Thinking": Reframing the Reasoning Cliff as an Agentic Gap
por: Khan, Sheraz, et al.
Publicado: (2025)

Reviving The Classics: Active Reward Modeling in Large Language Model Alignment
por: Shen, Yunyi, et al.
Publicado: (2025)

Simple Policy Gradients for Reasoning with Diffusion Language Models
por: Zhan, Anthony
Publicado: (2025)

Stepwise Alignment for Constrained Language Model Policy Optimization
por: Wachi, Akifumi, et al.
Publicado: (2024)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
por: Zhang, Songming, et al.
Publicado: (2025)

RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models
por: Feng, Xiao, et al.
Publicado: (2026)

PPO-BR: Dual-Signal Entropy-Reward Adaptation for Trust Region Policy Optimization
por: Rahman, Ben
Publicado: (2025)

PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning
por: Wu, Feijie, et al.
Publicado: (2025)

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
por: Du, Yihan, et al.
Publicado: (2024)

Rewarding Intellectual Humility Learning When Not To Answer In Large Language Models
por: Jha, Abha, et al.
Publicado: (2026)

DP-OPD: Differentially Private On-Policy Distillation for Language Models
por: Khadem, Fatemeh, et al.
Publicado: (2026)

Equivalent Linear Mappings of Large Language Models
por: Golden, James R.
Publicado: (2025)

Policy Learning with a Language Bottleneck
por: Srivastava, Megha, et al.
Publicado: (2024)

DavIR: Data Selection via Implicit Reward for Large Language Models
por: Zhou, Haotian, et al.
Publicado: (2023)

On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
por: Agarwal, Rishabh, et al.
Publicado: (2023)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
por: Heuillet, Maxime, et al.
Publicado: (2025)

A Relative-Budget Theory for Reinforcement Learning with Verifiable Rewards in Large Language Model Reasoning
por: Wachi, Akifumi, et al.
Publicado: (2026)

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
por: Sun, Hao, et al.
Publicado: (2025)

On the Robustness of Reward Models for Language Model Alignment
por: Hong, Jiwoo, et al.
Publicado: (2025)