Imagen de Portada

Intrinsic Mutual Information as a Modulator for Preference Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liao, Peng, Zheng, Peijia, Li, Lingbo, Liang, Shangsong, Chen, Lin
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Modular Retrieval-Augmented Generalization for Human Action Recognition
por: Liao, Peng, et al.
Publicado: (2026)

SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters
por: Xiao, Teng, et al.
Publicado: (2025)

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation
por: Qin, Peijia, et al.
Publicado: (2024)

D-SCoRE: Document-Centric Segmentation and CoT Reasoning with Structured Export for QA-CoT Data Generation
por: Zhou, Weibo, et al.
Publicado: (2025)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
por: Wang, Xiaobo, et al.
Publicado: (2025)

Orthogonal Finetuning for Direct Preference Optimization
por: Yang, Chenxu, et al.
Publicado: (2024)

Preference Optimization with Multi-Sample Comparisons
por: Wang, Chaoqi, et al.
Publicado: (2024)

Enhancing LLM Safety via Constrained Direct Preference Optimization
por: Liu, Zixuan, et al.
Publicado: (2024)

MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media
por: Manzoor, Muhammad Arslan, et al.
Publicado: (2024)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
por: Zhu, Zining, et al.
Publicado: (2025)

Modulated Intervention Preference Optimization (MIPO): Keep the Easy, Refine the Difficult
por: Jang, Cheolhun
Publicado: (2024)

SimPO: Simple Preference Optimization with a Reference-Free Reward
por: Meng, Yu, et al.
Publicado: (2024)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

What Level of Automation is "Good Enough"? A Benchmark of Large Language Models for Meta-Analysis Data Extraction
por: Li, Lingbo, et al.
Publicado: (2025)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
por: Zhang, Xuan, et al.
Publicado: (2024)

Optimizing Language Models for Human Preferences is a Causal Inference Problem
por: Lin, Victoria, et al.
Publicado: (2024)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
por: Lin, Yong, et al.
Publicado: (2024)

PORT: Preference Optimization on Reasoning Traces
por: Lahlou, Salem, et al.
Publicado: (2024)

Length Desensitization in Direct Preference Optimization
por: Liu, Wei, et al.
Publicado: (2024)

Efficient RLVR Training via Weighted Mutual Information Data Selection
por: Zhou, Xinyu, et al.
Publicado: (2026)

Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization
por: Cao, Yuanpu, et al.
Publicado: (2024)

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
por: He, Yinhan, et al.
Publicado: (2026)

ROPO: Robust Preference Optimization for Large Language Models
por: Liang, Xize, et al.
Publicado: (2024)

DreamPRM-Code: Function-as-Step Process Reward Model with Label Correction for LLM Coding
por: Zhang, Ruiyi, et al.
Publicado: (2025)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
por: Peng, Andi, et al.
Publicado: (2024)

Many of Your DPOs are Secretly One: Attempting Unification Through Mutual Information
por: Tutnov, Rasul, et al.
Publicado: (2025)

Understanding Reference Policies in Direct Preference Optimization
por: Liu, Yixin, et al.
Publicado: (2024)

Accelerating Direct Preference Optimization with Prefix Sharing
por: Wang, Franklin, et al.
Publicado: (2024)

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
por: Li, Junsong, et al.
Publicado: (2025)

AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
por: Chen, Shaolong, et al.
Publicado: (2026)

TSO: Self-Training with Scaled Preference Optimization
por: Chen, Kaihui, et al.
Publicado: (2024)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
por: Chen, Guanzheng, et al.
Publicado: (2025)

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
por: Xiao, Teng, et al.
Publicado: (2024)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
por: Lai, Xin, et al.
Publicado: (2024)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
por: Yin, Yueqin, et al.
Publicado: (2024)

Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation
por: Yao, Jiashu, et al.
Publicado: (2026)

STENCIL: Submodular Mutual Information Based Weak Supervision for Cold-Start Active Learning
por: Beck, Nathan, et al.
Publicado: (2024)

Group Robust Preference Optimization in Reward-free RLHF
por: Ramesh, Shyam Sundhar, et al.
Publicado: (2024)

Direct Multi-Turn Preference Optimization for Language Agents
por: Shi, Wentao, et al.
Publicado: (2024)

Multi-Reference Preference Optimization for Large Language Models
por: Le, Hung, et al.
Publicado: (2024)