Cover Image

KL Penalty Control via Perturbation for Direct Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lee, Sangkyu, Han, Janghoon, Song, Hosung, Choi, Stanley Jungkyu, Lee, Honglak, Yu, Youngjae
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Spanning Tree Autoregressive Visual Generation
by: Lee, Sangkyu, et al.
Published: (2025)

Mitigating Biases for Instruction-following Language Models via Bias Neurons Elimination
by: Yang, Nakyeong, et al.
Published: (2023)

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety
by: Kim, Geon-Hyeong, et al.
Published: (2025)

Clear Preferences Leave Traces: Reference Model-Guided Sampling for Preference Learning
by: Diwan, Nirav, et al.
Published: (2025)

Aligning Large Language Models by On-Policy Self-Judgment
by: Lee, Sangkyu, et al.
Published: (2024)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
by: Huang, Audrey, et al.
Published: (2024)

To Predict or Not To Predict? Proportionally Masked Autoencoders for Tabular Data Imputation
by: Kim, Jungkyu, et al.
Published: (2024)

KL for a KL: On-Policy Distillation with Control Variate Baseline
by: Oh, Minjae, et al.
Published: (2026)

Context information can be more important than reasoning for time series forecasting with a large language model
by: Yang, Janghoon
Published: (2025)

C2-DPO: Constrained Controlled Direct Preference Optimization
by: Asadi, Kavosh, et al.
Published: (2025)

Learning to Theorize the World from Observation
by: Baek, Doojin, et al.
Published: (2026)

Preference-Based Self-Distillation: Beyond KL Matching via Reward Regularization
by: Yu, Xin, et al.
Published: (2026)

Instruction Matters: A Simple yet Effective Task Selection for Optimized Instruction Tuning of Specific Tasks
by: Lee, Changho, et al.
Published: (2024)

Entropy Controllable Direct Preference Optimization
by: Omura, Motoki, et al.
Published: (2024)

Semi-Supervised Preference Optimization with Limited Feedback
by: Lee, Seonggyun, et al.
Published: (2025)

ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learning
by: Lee, Hosung, et al.
Published: (2024)

Boost Your Human Image Generation Model via Direct Preference Optimization
by: Na, Sanghyeon, et al.
Published: (2024)

Policy-labeled Preference Learning: Is Preference Enough for RLHF?
by: Cho, Taehyun, et al.
Published: (2025)

Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
by: Won, Yunjae, et al.
Published: (2025)

Optimal Stability of KL Divergence under Gaussian Perturbations
by: Pan, Jialu, et al.
Published: (2026)

ADPO: Anchored Direct Preference Optimization
by: Zixian, Wang
Published: (2025)

Continuous-Utility Direct Preference Optimization
by: Mohsin, Muhammad Ahmed, et al.
Published: (2026)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

$ξ$-DPO: Direct Preference Optimization via Ratio Reward Margin
by: Fan, Zhengyuan, et al.
Published: (2026)

DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation
by: Han, Janghoon, et al.
Published: (2025)

Risk-aware Direct Preference Optimization under Nested Risk Measure
by: Zhang, Lijun, et al.
Published: (2025)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning
by: Han, Janghoon, et al.
Published: (2024)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
by: Wu, Junkang, et al.
Published: (2024)

Filtered Direct Preference Optimization
by: Morimura, Tetsuro, et al.
Published: (2024)

Direct Preference Optimization with an Offset
by: Amini, Afra, et al.
Published: (2024)

Causal Disentanglement Learning for Accurate Anomaly Detection in Multivariate Time Series
by: Kim, Wonah, et al.
Published: (2025)

Thinking Preference Optimization
by: Yang, Wang, et al.
Published: (2025)

SPAP: Structured Pruning via Alternating Optimization and Penalty Methods
by: Hu, Hanyu, et al.
Published: (2025)

Self-Improving Robust Preference Optimization
by: Choi, Eugene, et al.
Published: (2024)

Mitigating Overfitting in Graph Neural Networks via Feature and Hyperplane Perturbation
by: Choi, Yoonhyuk, et al.
Published: (2022)

Adaptive Batch-Wise Sample Scheduling for Direct Preference Optimization
by: Huang, Zixuan, et al.
Published: (2025)