Cover Image

Value Imprint: A Technique for Auditing the Human Values Embedded in RLHF Datasets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Obi, Ike, Pant, Rohan, Agrawal, Srishti Shekhar, Ghazanfar, Maham, Basiletti, Aaron
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ROCM: RLHF on consistency models
by: Shekhar, Shivanshu, et al.
Published: (2025)

On the Performance of Imputation Techniques for Missing Values on Healthcare Datasets
by: Joel, Luke Oluwaseye, et al.
Published: (2024)

Rethinking KL Regularization in RLHF: From Value Estimation to Gradient Optimization
by: Liu, Kezhao, et al.
Published: (2025)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
by: Cen, Shicong, et al.
Published: (2024)

Phase-Aware Deep Learning with Complex-Valued CNNs for Audio Signal Applications
by: Agrawal, Naman
Published: (2025)

A Descriptive and Normative Theory of Human Beliefs in RLHF
by: Dandekar, Sylee, et al.
Published: (2025)

Dataset Reset Policy Optimization for RLHF
by: Chang, Jonathan D., et al.
Published: (2024)

floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
by: Agrawalla, Bhavya, et al.
Published: (2025)

Learning Human-like Representations to Enable Learning Human Values
by: Wynn, Andrea, et al.
Published: (2023)

Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving
by: Gao, Wei, et al.
Published: (2025)

WPO: Enhancing RLHF with Weighted Preference Optimization
by: Zhou, Wenxuan, et al.
Published: (2024)

Value Augmented Sampling for Language Model Alignment and Personalization
by: Han, Seungwook, et al.
Published: (2024)

Pretrain Value, Not Reward: Decoupled Value Policy Optimization
by: Huang, Chenghua, et al.
Published: (2025)

Value Flows
by: Dong, Perry, et al.
Published: (2025)

Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
by: Xu, Hefei, et al.
Published: (2025)

LLM-Assisted Logic Rule Learning: Scaling Human Expertise for Time Series Anomaly Detection
by: Zhang, Haoting, et al.
Published: (2026)

Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback
by: Ji, Jiaming, et al.
Published: (2025)

Influencing Humans to Conform to Preference Models for RLHF
by: Hatgis-Kessell, Stephane, et al.
Published: (2025)

An Analysis of Action-Value Temporal-Difference Methods That Learn State Values
by: Daley, Brett, et al.
Published: (2025)

Regression-adjusted Monte Carlo Estimators for Shapley Values and Probabilistic Values
by: Witter, R. Teal, et al.
Published: (2025)

RLHF Workflow: From Reward Modeling to Online RLHF
by: Dong, Hanze, et al.
Published: (2024)

Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison
by: Shen, Judy Hanwen, et al.
Published: (2024)

RVFL-X: A Novel Randomized Network Based on Complex Transformed Real-Valued Tabular Datasets
by: Sajid, M., et al.
Published: (2025)

Is there Value in Reinforcement Learning?
by: Fox, Lior, et al.
Published: (2025)

Distributionally Robust Token Optimization in RLHF
by: Jin, Yeping, et al.
Published: (2026)

AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
by: Lee, Dongsu, et al.
Published: (2024)

Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective
by: Kang, Yipeng, et al.
Published: (2024)

VC-Soup: Value-Consistency Guided Multi-Value Alignment for Large Language Models
by: Xu, Hefei, et al.
Published: (2026)

MaxMin-RLHF: Alignment with Diverse Human Preferences
by: Chakraborty, Souradip, et al.
Published: (2024)

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
by: Xiong, Wei, et al.
Published: (2023)

On the Inflation of KNN-Shapley Value
by: Yang, Ziao, et al.
Published: (2024)

Highway Value Iteration Networks
by: Wang, Yuhui, et al.
Published: (2024)

An Odd Estimator for Shapley Values
by: Fumagalli, Fabian, et al.
Published: (2026)

Significativity Indices for Agreement Values
by: Casagrande, Alberto, et al.
Published: (2025)

Universal Value-Function Uncertainties
by: Zanger, Moritz A., et al.
Published: (2025)

Suboptimal Shapley Value Explanations
by: Lu, Xiaolei
Published: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
by: Hu, Jian, et al.
Published: (2024)

Towards Reliable Alignment: Uncertainty-aware RLHF
by: Banerjee, Debangshu, et al.
Published: (2024)

Boosting Deductive Reasoning with Step Signals In RLHF
by: Li, Jialian, et al.
Published: (2024)

The Hidden Link Between RLHF and Contrastive Learning
by: Lv, Xufei, et al.
Published: (2025)