Imagen de Portada

Entropy Controllable Direct Preference Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Omura, Motoki, Fujita, Yasuhiro, Kataoka, Toshiki
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Filtered Direct Preference Optimization
por: Morimura, Tetsuro, et al.
Publicado: (2024)

Direct Preference Optimization with an Offset
por: Amini, Afra, et al.
Publicado: (2024)

Orthogonal Finetuning for Direct Preference Optimization
por: Yang, Chenxu, et al.
Publicado: (2024)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
por: Razin, Noam, et al.
Publicado: (2024)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
por: Wu, Junkang, et al.
Publicado: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
por: Wang, Haoxiang, et al.
Publicado: (2024)

Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing
por: Qi, Biqing, et al.
Publicado: (2024)

Inducing Robustness in a 2 Dimensional Direct Preference Optimization Paradigm
por: Shashidhar, Sarvesh, et al.
Publicado: (2025)

Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
por: Won, Yunjae, et al.
Publicado: (2025)

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
por: Rosset, Corby, et al.
Publicado: (2024)

A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications
por: Xiao, Wenyi, et al.
Publicado: (2024)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
por: Wu, Junkang, et al.
Publicado: (2024)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
por: Zhu, Mingkang, et al.
Publicado: (2025)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
por: Rafailov, Rafael, et al.
Publicado: (2023)

Mix- and MoE-DPO: A Variational Inference Approach to Direct Preference Optimization
por: Bohne, Jason, et al.
Publicado: (2025)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
por: Kim, Dongyoung, et al.
Publicado: (2024)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
por: Huang, Audrey, et al.
Publicado: (2024)

Geometric-Averaged Preference Optimization for Soft Preference Labels
por: Furuta, Hiroki, et al.
Publicado: (2024)

PLaMo-100B: A Ground-Up Language Model Designed for Japanese Proficiency
por: Elements, Preferred, et al.
Publicado: (2024)

GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
por: Simoni, Marco, et al.
Publicado: (2025)

Self-Consistency Preference Optimization
por: Prasad, Archiki, et al.
Publicado: (2024)

Towards Improved Preference Optimization Pipeline: from Data Generation to Budget-Controlled Regularization
por: Chen, Zhuotong, et al.
Publicado: (2024)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
por: Jian, Chengtao, et al.
Publicado: (2025)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
por: Gupta, Taneesh, et al.
Publicado: (2025)

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
por: Ota, Kazuki, et al.
Publicado: (2026)

Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
por: Liu, Jie, et al.
Publicado: (2024)

Active Preference Optimization for Sample Efficient RLHF
por: Das, Nirjhar, et al.
Publicado: (2024)

VPO: Leveraging the Number of Votes in Preference Optimization
por: Cho, Jae Hyeon, et al.
Publicado: (2024)

TSO: Self-Training with Scaled Preference Optimization
por: Chen, Kaihui, et al.
Publicado: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

Preference Optimization by Estimating the Ratio of the Data Distribution
por: Kim, Yeongmin, et al.
Publicado: (2025)

ROPO: Robust Preference Optimization for Large Language Models
por: Liang, Xize, et al.
Publicado: (2024)

Accelerated Preference Optimization for Large Language Model Alignment
por: He, Jiafan, et al.
Publicado: (2024)

Self-Play Preference Optimization for Language Model Alignment
por: Wu, Yue, et al.
Publicado: (2024)

T-REG: Preference Optimization with Token-Level Reward Regularization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning
por: Fan, Chongyu, et al.
Publicado: (2024)

Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
por: D'Oosterlinck, Karel, et al.
Publicado: (2024)

Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning
por: Zhang, Ruiqi, et al.
Publicado: (2024)

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
por: Wang, Jialu, et al.
Publicado: (2026)