Cover Image

$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Jiaqi, Jiang, Mingjian, Song, Yuxuan, Ermon, Stefano, Xu, Minkai
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Divergence Minimization Preference Optimization for Diffusion Model Alignment
by: Li, Binxu, et al.
Published: (2025)

RFG: Test-Time Scaling for Diffusion Large Language Model Reasoning with Reward-Free Guidance
by: Chen, Tianlang, et al.
Published: (2025)

Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective
by: Ou, Jingyang, et al.
Published: (2025)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

Energy-Based Diffusion Language Models for Text Generation
by: Xu, Minkai, et al.
Published: (2024)

Geometric Trajectory Diffusion Models
by: Han, Jiaqi, et al.
Published: (2024)

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
by: She, Shuaijie, et al.
Published: (2025)

Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding
by: Guo, Gabe, et al.
Published: (2025)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Variational f-divergence Minimization
by: Zhang, Mingtian, et al.
Published: (2019)

Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
by: Shen, Yangyi, et al.
Published: (2026)

LiPO: Listwise Preference Optimization through Learning-to-Rank
by: Liu, Tianqi, et al.
Published: (2024)

SimPO: Simple Preference Optimization with a Reference-Free Reward
by: Meng, Yu, et al.
Published: (2024)

Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement
by: Dang, Meihua, et al.
Published: (2025)

Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
by: Lou, Aaron, et al.
Published: (2023)

Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
by: Wang, Austin, et al.
Published: (2026)

Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation
by: Manvi, Rohin, et al.
Published: (2024)

CHORDS: Diffusion Sampling Accelerator with Multi-core Hierarchical ODE Solvers
by: Han, Jiaqi, et al.
Published: (2025)

FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Users
by: Singh, Anikait, et al.
Published: (2025)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
by: Zhu, Zining, et al.
Published: (2025)

Align Your Structures: Generating Trajectories with Structure Pretraining for Molecular Dynamics
by: Iyengar, Aniketh, et al.
Published: (2026)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
by: Wang, Xiaobo, et al.
Published: (2025)

RankPO: Preference Optimization for Job-Talent Matching
by: Zhang, Yafei, et al.
Published: (2025)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
by: Chen, Guanzheng, et al.
Published: (2025)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
by: Yang, Ling, et al.
Published: (2024)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
by: Yoon, Hee Suk, et al.
Published: (2025)

ComPO: Preference Alignment via Comparison Oracles
by: Chen, Peter, et al.
Published: (2025)

RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization
by: Liu, Tianci, et al.
Published: (2025)

GeoAda: Efficiently Finetune Geometric Diffusion Models with Equivariant Adapters
by: Zhao, Wanjia, et al.
Published: (2025)

GeoLLM: Extracting Geospatial Knowledge from Large Language Models
by: Manvi, Rohin, et al.
Published: (2023)

Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants
by: Jalouzot, Louis, et al.
Published: (2025)

TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation
by: Shi, Juntong, et al.
Published: (2024)

Contextualized Diffusion Models for Text-Guided Image and Video Generation
by: Yang, Ling, et al.
Published: (2024)

Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model
by: Hong, Yuzhong, et al.
Published: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
by: Zhou, Wenxuan, et al.
Published: (2024)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
by: Sun, Shengyang, et al.
Published: (2025)

SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks
by: Christopoulou, Fenia, et al.
Published: (2024)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

Putting It All into Context: Simplifying Agents with LCLMs
by: Jiang, Mingjian, et al.
Published: (2025)