Image de couverture de livre

Solving the Inverse Alignment Problem for Efficient RLHF

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Krishna, Shambhavi, Sahoo, Aishwarya
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Why Is RLHF Alignment Shallow? A Gradient Analysis
par: Young, Robin
Publié: (2026)

RLHF Workflow: From Reward Modeling to Online RLHF
par: Dong, Hanze, et autres
Publié: (2024)

Learning from Failures: Understanding LLM Alignment through Failure-Aware Inverse RL
par: Patel, Nyal, et autres
Publié: (2025)

MaxMin-RLHF: Alignment with Diverse Human Preferences
par: Chakraborty, Souradip, et autres
Publié: (2024)

Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning
par: Krishna, Shambhavi, et autres
Publié: (2025)

Proxy-RLHF: Decoupling Generation and Alignment in Large Language Model with Proxy
par: Zhu, Yu, et autres
Publié: (2024)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
par: Du, Yuhao, et autres
Publié: (2025)

Active Preference Optimization for Sample Efficient RLHF
par: Das, Nirjhar, et autres
Publié: (2024)

RLHF: A comprehensive Survey for Cultural, Multimodal and Low Latency Alignment Methods
par: Sharma, Raghav, et autres
Publié: (2025)

Failure Modes of Maximum Entropy RLHF
par: Çağatan, Ömer Veysel, et autres
Publié: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
par: Hu, Jian, et autres
Publié: (2024)

Evaluating Defences against Unsafe Feedback in RLHF
par: Rosati, Domenic, et autres
Publié: (2024)

FCoReBench: Can Large Language Models Solve Challenging First-Order Combinatorial Reasoning Problems?
par: Mittal, Chinmay, et autres
Publié: (2024)

Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF
par: Gao, Zhaolin, et autres
Publié: (2024)

Group Robust Preference Optimization in Reward-free RLHF
par: Ramesh, Shyam Sundhar, et autres
Publié: (2024)

Leveraging Domain Knowledge for Efficient Reward Modelling in RLHF: A Case-Study in E-Commerce Opinion Summarization
par: Nath, Swaroop, et autres
Publié: (2024)

Reward-Robust RLHF in LLMs
par: Yan, Yuzi, et autres
Publié: (2024)

RLHF and IIA: Perverse Incentives
par: Xu, Wanqiao, et autres
Publié: (2023)

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
par: Noukhovitch, Michael, et autres
Publié: (2024)

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF
par: Xie, Tengyang, et autres
Publié: (2024)

Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
par: Du, Yihan, et autres
Publié: (2024)

A Long Way to Go: Investigating Length Correlations in RLHF
par: Singhal, Prasann, et autres
Publié: (2023)

Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions
par: Patel, Dhruvesh, et autres
Publié: (2025)

Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
par: Hou, Zhenyu, et autres
Publié: (2024)

Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
par: Shi, Ruizhe, et autres
Publié: (2025)

How to Evaluate Reward Models for RLHF
par: Frick, Evan, et autres
Publié: (2024)

Dataset Reset Policy Optimization for RLHF
par: Chang, Jonathan D., et autres
Publié: (2024)

Measuring memorization in RLHF for code completion
par: Pappu, Aneesh, et autres
Publié: (2024)

Reward Model Overoptimisation in Iterated RLHF
par: Wolf, Lorenz, et autres
Publié: (2025)

On Importance of Pruning and Distillation for Efficient Low Resource NLP
par: Mirashi, Aishwarya, et autres
Publié: (2024)

Disentangled Safety Adapters Enable Efficient Guardrails and Flexible Inference-Time Alignment
par: Krishna, Kundan, et autres
Publié: (2025)

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs
par: Wang, Ruoyu, et autres
Publié: (2024)

Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs
par: Hu, Zhiyuan, et autres
Publié: (2026)

XFinBench: Benchmarking LLMs in Complex Financial Problem Solving and Reasoning
par: Zhang, Zhihan, et autres
Publié: (2025)

Quantile Regression for Distributional Reward Models in RLHF
par: Dorka, Nicolai
Publié: (2024)

The Perfect Blend: Redefining RLHF with Mixture of Judges
par: Xu, Tengyu, et autres
Publié: (2024)

ODIN: Disentangled Reward Mitigates Hacking in RLHF
par: Chen, Lichang, et autres
Publié: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
par: Zhou, Wenxuan, et autres
Publié: (2024)

Understanding the Effects of RLHF on LLM Generalisation and Diversity
par: Kirk, Robert, et autres
Publié: (2023)

RLHS: Mitigating Misalignment in RLHF with Hindsight Simulation
par: Liang, Kaiqu, et autres
Publié: (2025)