Image de couverture de livre

Mitigating Mismatch within Reference-based Preference Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yuan, Suqin, Yu, Xingrui, Zheng, Jiyang, Feng, Lei, Wang, Dadong, Tsang, Ivor, Liu, Tongliang
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
par: Zhao, Heyang, et autres
Publié: (2025)

Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Exploration
par: Yu, Xingrui, et autres
Publié: (2024)

Sharpness-Aware Black-Box Optimization
par: Ye, Feiyang, et autres
Publié: (2024)

Early Stopping Against Label Noise Without Validation Data
par: Yuan, Suqin, et autres
Publié: (2025)

Diversifying Policy Behaviors with Extrinsic Behavioral Curiosity
par: Wan, Zhenglin, et autres
Publié: (2024)

Distributional Multi-objective Black-box Optimization for Diffusion-model Inference-time Multi-Target Generation
par: Tan, Kim Yong, et autres
Publié: (2025)

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed
par: Dang, Sizhe, et autres
Publié: (2025)

Mitigating Preference Hacking in Policy Optimization with Pessimism
par: Gupta, Dhawal, et autres
Publié: (2025)

Enhancing Sample Selection Against Label Noise by Cutting Mislabeled Easy Examples
par: Yuan, Suqin, et autres
Publié: (2025)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
par: Liu, Qi, et autres
Publié: (2025)

Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Field
par: Tan, Kim Yong, et autres
Publié: (2026)

Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generation
par: Tan, Kim Yong, et autres
Publié: (2025)

Intelligently Weighting Multiple Reference Models for Direct Preference Optimization of LLMs
par: Wu, Skyler, et autres
Publié: (2025)

Radiology Report Generation via Multi-objective Preference Optimization
par: Xiao, Ting, et autres
Publié: (2024)

Hindsight Preference Optimization for Financial Time Series Advisory
par: Cui, Yanwei, et autres
Publié: (2026)

Olaf-World: Orienting Latent Actions for Video World Modeling
par: Jiang, Yuxin, et autres
Publié: (2026)

Beyond Precision: Training-Inference Mismatch is an Optimization Problem and Simple LR Scheduling Fixes It
par: Zhang, Yaxiang, et autres
Publié: (2026)

Unifying Stable Optimization and Reference Regularization in RLHF
par: He, Li, et autres
Publié: (2026)

Transductive Reward Inference on Graph
par: Qu, Bohao, et autres
Publié: (2024)

Spurious Correlation Learning in Preference Optimization: Mechanisms, Consequences, and Mitigation via Tie Training
par: Moya, Christian, et autres
Publié: (2026)

Instance-dependent Early Stopping
par: Yuan, Suqin, et autres
Publié: (2025)

Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transport
par: Wang, Zhanpeng, et autres
Publié: (2024)

Sem-DPO: Mitigating Semantic Inconsistency in Preference Optimization for Prompt Engineering
par: Mohamed, Anas, et autres
Publié: (2025)

Thinking Preference Optimization
par: Yang, Wang, et autres
Publié: (2025)

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning
par: Diwan, Nirav, et autres
Publié: (2025)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
par: Zhou, Zhanhui, et autres
Publié: (2023)

Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers
par: Cai, Xin-Qiang, et autres
Publié: (2025)

Planning Under Observation Mismatch for Traffic Signal Control via Adaptive Modular World Models
par: Huang, Zherui, et autres
Publié: (2025)

ADPO: Anchored Direct Preference Optimization
par: Zixian, Wang
Publié: (2025)

APO: Alpha-Divergence Preference Optimization
par: Zixian, Wang
Publié: (2025)

Revealing Multimodal Causality with Large Language Models
par: Li, Jin, et autres
Publié: (2025)

Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents
par: Gao, Heyang, et autres
Publié: (2025)

Conceptual Belief-Informed Reinforcement Learning
par: Gu, Xingrui, et autres
Publié: (2024)

Dual-Balancing for Multi-Task Learning
par: Lin, Baijiong, et autres
Publié: (2023)

Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction
par: Zhang, Ziru, et autres
Publié: (2025)

PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learning
par: Sun, Shengjie, et autres
Publié: (2025)

EnerBridge-DPO: Energy-Guided Protein Inverse Folding with Markov Bridges and Direct Preference Optimization
par: Rong, Dingyi, et autres
Publié: (2025)

Hindsight Preference Learning for Offline Preference-based Reinforcement Learning
par: Gao, Chen-Xiao, et autres
Publié: (2024)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
par: Yang, Jinming, et autres
Publié: (2026)

Orthogonal Finetuning for Direct Preference Optimization
par: Yang, Chenxu, et autres
Publié: (2024)