Cover Image

Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rita, Mathieu, Strub, Florian, Chaabouni, Rahma, Michel, Paul, Dupoux, Emmanuel, Pietquin, Olivier
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Language Evolution with Deep Learning
by: Rita, Mathieu, et al.
Published: (2024)

Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approach
by: Poli, Maxime, et al.
Published: (2024)

fastabx: A library for efficient computation of ABX discriminability
by: Poli, Maxime, et al.
Published: (2025)

Reinforcement Learning Amplifies Emergent Misalignment from Harmless Rewards
by: Jørgenvåg, Magnus, et al.
Published: (2026)

Simulating Articulatory Trajectories with Phonological Feature Interpolation
by: Tandazo, Angelo Ortiz, et al.
Published: (2024)

Enhanced LLM Reasoning by Optimizing Reward Functions with Search-Driven Reinforcement Learning
by: Ahmadi, Arash, et al.
Published: (2026)

R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning
by: Zhao, Qingfei, et al.
Published: (2025)

Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
by: Cheng, Ruoxi, et al.
Published: (2025)

OpenReward: Learning to Reward Long-form Agentic Tasks via Reinforcement Learning
by: Hu, Ziyou, et al.
Published: (2025)

Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning
by: Zhang, Kongcheng, et al.
Published: (2025)

EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models
by: de Seyssel, Maureen, et al.
Published: (2023)

LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning
by: Singh, Utsav, et al.
Published: (2024)

Agentic Reinforcement Learning with Implicit Step Rewards
by: Liu, Xiaoqian, et al.
Published: (2025)

Reward-Guided Speculative Decoding for Efficient LLM Reasoning
by: Liao, Baohao, et al.
Published: (2025)

Misconfidence-based Demonstration Selection for LLM In-Context Learning
by: Xu, Shangqing, et al.
Published: (2024)

MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery
by: Tandazo, Angelo Ortiz, et al.
Published: (2025)

DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Units
by: Poli, Maxime, et al.
Published: (2026)

Demonstration Selection for In-Context Learning via Reinforcement Learning
by: Wang, Xubin, et al.
Published: (2024)

LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble
by: Lee, Yujeong, et al.
Published: (2024)

BAGEL: Benchmarking Animal Knowledge Expertise in Language Models
by: Shen, Jiacheng, et al.
Published: (2026)

xJailbreak: Representation Space Guided Reinforcement Learning for Interpretable LLM Jailbreaking
by: Lee, Sunbowen, et al.
Published: (2025)

Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards
by: Lara, Luis, et al.
Published: (2026)

Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards
by: Tang, Xinyu, et al.
Published: (2025)

HalluCounter: Reference-free LLM Hallucination Detection in the Wild!
by: Urlana, Ashok, et al.
Published: (2025)

Beyond Imitation: Recovering Dense Rewards from Demonstrations
by: Li, Jiangnan, et al.
Published: (2025)

Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training
by: Xu, Ran, et al.
Published: (2026)

Distribution-Aware Reward: Reinforcement Learning over Predictive Distributions for LLM Regression
by: Park, Jungsoo, et al.
Published: (2026)

"Reasoning" with Rhetoric: On the Style-Evidence Tradeoff in LLM-Generated Counter-Arguments
by: Verma, Preetika, et al.
Published: (2024)

IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards
by: Guo, Xu, et al.
Published: (2025)

Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning
by: Du, Bodong, et al.
Published: (2026)

Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation
by: Min, Do June, et al.
Published: (2024)

Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signals
by: Chen, Sirui, et al.
Published: (2026)

TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
by: Wu, Jinyang, et al.
Published: (2025)

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
by: Zhang, Dan, et al.
Published: (2024)

R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning
by: Zhang, Yi-Fan, et al.
Published: (2025)

A LLM-Based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation
by: Zubiaga, Irune, et al.
Published: (2024)

Self-Guided Process Reward Optimization with Redefined Step-wise Advantage for Process Reinforcement Learning
by: Fei, Wu, et al.
Published: (2025)

Enhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding
by: Liao, Kuo, et al.
Published: (2024)

RLSR: Reinforcement Learning with Supervised Reward Outperforms SFT in Instruction Following
by: Wang, Zhichao, et al.
Published: (2025)

Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation
by: Liu, Yifeng, et al.
Published: (2026)