Cover Image

Refining Alignment Framework for Diffusion Models with Intermediate-Step Preference Ranking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ren, Jie, Zhang, Yuhang, Liu, Dongrui, Zhang, Xiaopeng, Tian, Qi
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learning
by: Ding, Zihan, et al.
Published: (2024)

Preference-Based Alignment of Discrete Diffusion Models
by: Borso, Umberto, et al.
Published: (2025)

Step-level Denoising-time Diffusion Alignment with Multiple Objectives
by: Zhang, Qi, et al.
Published: (2026)

SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Models
by: Qin, You, et al.
Published: (2026)

Latent Embedding Adaptation for Human Preference Alignment in Diffusion Planners
by: Ng, Wen Zheng Terence, et al.
Published: (2025)

Time Series Forecasting via Direct Per-Step Probability Distribution Modeling
by: Kong, Linghao, et al.
Published: (2025)

Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principles
by: Wei, Qingyan, et al.
Published: (2025)

Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation
by: Liu, Xiaotian, et al.
Published: (2026)

Structure-based RNA Design by Step-wise Optimization of Latent Diffusion Model
by: Si, Qi, et al.
Published: (2026)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
by: Gao, Mingqi, et al.
Published: (2024)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

Teaching Your Models to Understand Code via Focal Preference Alignment
by: Wu, Jie, et al.
Published: (2025)

Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
by: Zhang, Shenao, et al.
Published: (2024)

Preference Learning Algorithms Do Not Learn Preference Rankings
by: Chen, Angelica, et al.
Published: (2024)

Adversarial Preference Learning for Robust LLM Alignment
by: Wang, Yuanfu, et al.
Published: (2025)

When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop
by: Zhang, Yang, et al.
Published: (2026)

Two-Step Offline Preference-Based Reinforcement Learning with Constrained Actions
by: Xu, Yinglun, et al.
Published: (2023)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
by: Pattnaik, Pulkit, et al.
Published: (2024)

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
by: Liang, Ren-Wei, et al.
Published: (2025)

BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning
by: Zhang, Beichen, et al.
Published: (2025)

OrdMoE: Preference Alignment via Hierarchical Expert Group Ranking in Multimodal Mixture-of-Experts LLMs
by: Gao, Yuting, et al.
Published: (2025)

Diffusion Model with Representation Alignment for Protein Inverse Folding
by: Wang, Chenglin, et al.
Published: (2024)

Inference-Time Alignment Control for Diffusion Models with Reinforcement Learning Guidance
by: Jin, Luozhijie, et al.
Published: (2025)

Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment
by: Zhang, Yifan, et al.
Published: (2024)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
by: Lai, Xin, et al.
Published: (2024)

Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
by: Huang, Wei, et al.
Published: (2026)

Learning Permutation Distributions via Reflected Diffusion on Ranks
by: He, Sizhuang, et al.
Published: (2026)

Energy-Structured Low-Rank Adaptation for Continual Learning
by: Li, Longhua, et al.
Published: (2026)

Direct Alignment with Heterogeneous Preferences
by: Shirali, Ali, et al.
Published: (2025)

Efficient Exploration for Iterative Nash Preference Optimization
by: Nan, Tianlong, et al.
Published: (2026)

Fine-Tuning Diffusion Models via Intermediate Distribution Shaping
by: Anil, Gautham Govind, et al.
Published: (2025)

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
by: Zhang, Yulin, et al.
Published: (2026)

PIPA: Preference Alignment as Prior-Informed Statistical Estimation
by: Li, Junbo, et al.
Published: (2025)

Human Alignment of Large Language Models through Online Preference Optimisation
by: Calandriello, Daniele, et al.
Published: (2024)

SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
by: Ren, Yanwei, et al.
Published: (2025)

EMSEdit: Efficient Multi-Step Meta-Learning-based Model Editing
by: Li, Xiaopeng, et al.
Published: (2025)

Calibrated Preference Learning: The Case of Label Ranking
by: Thies, Santo M. A. R., et al.
Published: (2026)

Post Hoc Regression Refinement via Pairwise Rankings
by: Wijaya, Kevin Tirta, et al.
Published: (2025)

InSPO: Unlocking Intrinsic Self-Reflection for LLM Preference Optimization
by: Li, Yu, et al.
Published: (2025)