Cover Image

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Belakaria, Syrine, Kazdan, Joshua, Marx, Charles, Cundy, Chris, Neiswanger, Willie, Koyejo, Sanmi, Engelhardt, Barbara E., Ermon, Stefano
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Preference-Guided Diffusion for Multi-Objective Offline Optimization
by: Annadani, Yashas, et al.
Published: (2025)

Sample Efficient Preference Alignment in LLMs via Active Exploration
by: Mehta, Viraj, et al.
Published: (2023)

Non-Myopic Multi-Objective Bayesian Optimization
by: Belakaria, Syrine, et al.
Published: (2024)

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes
by: Belakaria, Syrine, et al.
Published: (2024)

Neural Nonmyopic Bayesian Optimization in Dynamic Cost Settings
by: Truong, Sang T., et al.
Published: (2026)

SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking
by: Cundy, Chris, et al.
Published: (2023)

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms
by: Kazdan, Joshua, et al.
Published: (2025)

The Utility and Complexity of in- and out-of-Distribution Machine Unlearning
by: Allouah, Youssef, et al.
Published: (2024)

Probabilistic Graphical Models: A Concise Tutorial
by: Maasch, Jacqueline, et al.
Published: (2025)

Position: Model Collapse Does Not Mean What You Think
by: Schaeffer, Rylan, et al.
Published: (2025)

KGGen: Extracting Knowledge Graphs from Plain Text with Language Models
by: Mo, Belinda, et al.
Published: (2025)

Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients
by: Cundy, Chris, et al.
Published: (2020)

Pareto Front-Diverse Batch Multi-Objective Bayesian Optimization
by: Ahmadianshalchi, Alaleh, et al.
Published: (2024)

Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution
by: Wu, Tailin, et al.
Published: (2024)

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
by: Gupta, Isha, et al.
Published: (2025)

Preference Learning with Lie Detectors can Induce Honesty or Evasion
by: Cundy, Chris, et al.
Published: (2025)

CPSample: Classifier Protected Sampling for Guarding Training Data During Diffusion
by: Kazdan, Joshua, et al.
Published: (2024)

Efficient Prediction of Pass@k Scaling in Large Language Models
by: Kazdan, Joshua, et al.
Published: (2025)

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
by: Kazdan, Joshua, et al.
Published: (2024)

Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
by: Denisov-Blanch, Yegor, et al.
Published: (2026)

Calibrated Probabilistic Forecasts for Arbitrary Sequences
by: Marx, Charles, et al.
Published: (2024)

Active Preference Optimization for Sample Efficient RLHF
by: Das, Nirjhar, et al.
Published: (2024)

Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds
by: Chen, Edward, et al.
Published: (2025)

Latent Adversarial Regularization for Offline Preference Optimization
by: Jiang, Enyi, et al.
Published: (2026)

Causally Inspired Regularization Enables Domain General Representations
by: Salaudeen, Olawale, et al.
Published: (2024)

Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes
by: Robertson, Zachary, et al.
Published: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
by: Vo, Truong, et al.
Published: (2025)

Scale Dependent Data Duplication
by: Kazdan, Joshua, et al.
Published: (2026)

In-Context Learning of Energy Functions
by: Schaeffer, Rylan, et al.
Published: (2024)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

A Framework for Objective-Driven Dynamical Stochastic Fields
by: Zhang, Yibo Jacky, et al.
Published: (2025)

Rethinking RL for LLM Reasoning: It's Sparse Policy Selection, Not Capability Learning
by: Akgül, Ömer Faruk, et al.
Published: (2026)

How Do Large Language Monkeys Get Their Power (Laws)?
by: Schaeffer, Rylan, et al.
Published: (2025)

Efficient Evaluation of Multi-Task Robot Policies With Active Experiment Selection
by: Anwar, Abrar, et al.
Published: (2025)

Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
by: Lou, Aaron, et al.
Published: (2023)

The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior
by: Wang, Angelina, et al.
Published: (2025)

DeLLMa: Decision Making Under Uncertainty with Large Language Models
by: Liu, Ollie, et al.
Published: (2024)

Optimization and Generalization Guarantees for Weight Normalization
by: Cisneros-Velarde, Pedro, et al.
Published: (2024)

General Preference Reinforcement Learning
by: Umer, Muhammad, et al.
Published: (2026)

Divergence Minimization Preference Optimization for Diffusion Model Alignment
by: Li, Binxu, et al.
Published: (2025)