Cover Image

Group Robust Preference Optimization in Reward-free RLHF

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ramesh, Shyam Sundhar, Hu, Yifan, Chaimalas, Iason, Mehta, Viraj, Sessa, Pier Giuseppe, Ammar, Haitham Bou, Bogunovic, Ilija
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Distributionally Robust Model-based Reinforcement Learning with Large State Spaces
by: Ramesh, Shyam Sundhar, et al.
Published: (2023)

On Almost Surely Safe Alignment of Large Language Models at Inference-Time
by: Ji, Xiaotong, et al.
Published: (2025)

Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
by: Ramesh, Shyam Sundhar, et al.
Published: (2026)

Robust Multi-Objective Controlled Decoding of Large Language Models
by: Son, Seongho, et al.
Published: (2025)

LLM-WikiRace Benchmark: How Far Can LLMs Plan over Real-World Knowledge Graphs?
by: Ziomek, Juliusz, et al.
Published: (2026)

This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMs
by: Wolf, Lorenz, et al.
Published: (2025)

Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information
by: Tutnov, Rasul, et al.
Published: (2025)

SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks
by: Christopoulou, Fenia, et al.
Published: (2024)

Explorer: Robust Collection of Interactable GUI Elements
by: Chaimalas, Iason, et al.
Published: (2025)

Overton Pluralistic Reinforcement Learning for Large Language Models
by: Fu, Yu, et al.
Published: (2026)

Reward-Robust RLHF in LLMs
by: Yan, Yuzi, et al.
Published: (2024)

Why Can Large Language Models Generate Correct Chain-of-Thoughts?
by: Tutunov, Rasul, et al.
Published: (2023)

Reward Difference Optimization For Sample Reweighting In Offline RLHF
by: Wang, Shiqi, et al.
Published: (2024)

Active Preference Optimization for Sample Efficient RLHF
by: Das, Nirjhar, et al.
Published: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
by: Zhou, Wenxuan, et al.
Published: (2024)

Mixture of Attentions For Speculative Decoding
by: Zimmer, Matthieu, et al.
Published: (2024)

RLHF Workflow: From Reward Modeling to Online RLHF
by: Dong, Hanze, et al.
Published: (2024)

Risk-Controlled Lean-as-Judge for Natural-Language Mathematical Reasoning
by: Bourigault, Pauline, et al.
Published: (2026)

Mitigating Reward Hacking in RLHF via Advantage Sign Robustness
by: Ono, Shinnosuke, et al.
Published: (2026)

Subjective Depth and Timescale Transformers: Learning Where and When to Compute
by: Wieser, Frederico, et al.
Published: (2025)

YFPO: A Preliminary Study of Yoked Feature Preference Optimization with Neuron-Guided Rewards for Mathematical Reasoning
by: Le, Yifan
Published: (2026)

Robust Preference Optimization through Reward Model Distillation
by: Fisch, Adam, et al.
Published: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
by: Fu, Jiayi, et al.
Published: (2025)

Taming Overconfidence in LLMs: Reward Calibration in RLHF
by: Leng, Jixuan, et al.
Published: (2024)

Untangling Component Imbalance in Hybrid Linear Attention Conversion Methods
by: Benfeghoul, Martin, et al.
Published: (2025)

Robust Bayesian Optimisation with Unbounded Corruptions
by: Ezzerg, Abdelhamid, et al.
Published: (2025)

Reward Model Overoptimisation in Iterated RLHF
by: Wolf, Lorenz, et al.
Published: (2025)

How to Evaluate Reward Models for RLHF
by: Frick, Evan, et al.
Published: (2024)

Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
by: Son, Seongho, et al.
Published: (2024)

Prototypical Reward Network for Data-Efficient RLHF
by: Zhang, Jinghan, et al.
Published: (2024)

No Preference Left Behind: Group Distributional Preference Optimization
by: Yao, Binwei, et al.
Published: (2024)

Sample Efficient Preference Alignment in LLMs via Active Exploration
by: Mehta, Viraj, et al.
Published: (2023)

Adaptive Margin RLHF via Preference over Preferences
by: Chittepu, Yaswanth, et al.
Published: (2025)

SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning
by: Hazard, Hugo, et al.
Published: (2025)

Quantile Regression for Distributional Reward Models in RLHF
by: Dorka, Nicolai
Published: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
by: Chen, Lichang, et al.
Published: (2024)

Information-Theoretic Reward Decomposition for Generalizable RLHF
by: Mao, Liyuan, et al.
Published: (2025)

Al-Khwarizmi: Discovering Physical Laws with Foundation Models
by: Mower, Christopher E., et al.
Published: (2025)

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF
by: Xie, Tengyang, et al.
Published: (2024)

RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
by: Dang, John, et al.
Published: (2024)