Imagen de Portada

When Human Preferences Flip: An Instance-Dependent Robust Loss for RLHF

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Yifan, Ye, Xichen, Chen, Yifan, Zhang, Qiaosheng
Formato:	Preprint
Publicado:	2025
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Influencing Humans to Conform to Preference Models for RLHF
por: Hatgis-Kessell, Stephane, et al.
Publicado: (2025)

Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
por: Xiong, Wei, et al.
Publicado: (2023)

Single Agent Robust Deep Reinforcement Learning for Bus Fleet Control
por: Zhang, Yifan
Publicado: (2025)

MaxMin-RLHF: Alignment with Diverse Human Preferences
por: Chakraborty, Souradip, et al.
Publicado: (2024)

PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference
por: Ji, Jiaming, et al.
Publicado: (2024)

Policy-labeled Preference Learning: Is Preference Enough for RLHF?
por: Cho, Taehyun, et al.
Publicado: (2025)

WPO: Enhancing RLHF with Weighted Preference Optimization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

Generative RLHF-V: Learning Principles from Multi-modal Human Preference
por: Zhou, Jiayi, et al.
Publicado: (2025)

How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
por: Xu, Yifan, et al.
Publicado: (2026)

Adaptive Margin RLHF via Preference over Preferences
por: Chittepu, Yaswanth, et al.
Publicado: (2025)

Avoiding $\mathbf{exp(R_{max})}$ scaling in RLHF through Preference-based Exploration
por: Chen, Mingyu, et al.
Publicado: (2025)

RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation
por: Park, Chanwoo, et al.
Publicado: (2024)

RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoning
por: Wei, Zeming, et al.
Publicado: (2026)

Reward-Robust RLHF in LLMs
por: Yan, Yuzi, et al.
Publicado: (2024)

Active Preference Optimization for Sample Efficient RLHF
por: Das, Nirjhar, et al.
Publicado: (2024)

On Symmetric Losses for Robust Policy Optimization with Noisy Preferences
por: Nishimori, Soichiro, et al.
Publicado: (2025)

A Rational Model of Dimension-reduced Human Categorization
por: Hong, Yifan, et al.
Publicado: (2023)

Distributionally Robust Token Optimization in RLHF
por: Jin, Yeping, et al.
Publicado: (2026)

Active Negative Loss: A Robust Framework for Learning with Noisy Labels
por: Ye, Xichen, et al.
Publicado: (2024)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
por: Chen, Yanjun, et al.
Publicado: (2024)

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
por: Belakaria, Syrine, et al.
Publicado: (2025)

More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness
por: Li, Aaron J., et al.
Publicado: (2024)

Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
por: Siththaranjan, Anand, et al.
Publicado: (2023)

Trust, Don't Trust, or Flip: Robust Preference-Based Reinforcement Learning with Multi-Expert Feedback
por: Hosseini, Seyed Amir, et al.
Publicado: (2026)

Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks
por: Sun, Ruopei, et al.
Publicado: (2025)

APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMs
por: Srewa, Mahmoud, et al.
Publicado: (2026)

Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer
por: Liu, Zhihan, et al.
Publicado: (2024)

A Single-Point Measurement Framework for Robust Cyber-Attack Diagnosis in Smart Microgrids Using Dual Fractional-Order Feature Analysis
por: Wang, Yifan
Publicado: (2025)

Democratic Preference Alignment via Sortition-Weighted RLHF
por: Sana, Suvadip, et al.
Publicado: (2026)

Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation
por: Yu, Chengzhi, et al.
Publicado: (2025)

A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs
por: Srewa, Mahmoud, et al.
Publicado: (2025)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
por: Cen, Shicong, et al.
Publicado: (2024)

Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
por: Zhang, Yifan, et al.
Publicado: (2024)

When Truthful Representations Flip Under Deceptive Instructions?
por: Long, Xianxuan, et al.
Publicado: (2025)

A Descriptive and Normative Theory of Human Beliefs in RLHF
por: Dandekar, Sylee, et al.
Publicado: (2025)

Strong Preferences Affect the Robustness of Preference Models and Value Alignment
por: Xu, Ziwei, et al.
Publicado: (2024)

MicroNAS: Zero-Shot Neural Architecture Search for MCUs
por: Qiao, Ye, et al.
Publicado: (2024)

Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks
por: Hu, Rui, et al.
Publicado: (2024)

Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison
por: Shen, Judy Hanwen, et al.
Publicado: (2024)

SETransformer: A Hybrid Attention-Based Architecture for Robust Human Activity Recognition
por: Liu, Yunbo, et al.
Publicado: (2025)