Cover Image

DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Ruizhe, Chai, Wenhao, Yang, Zhifei, Zhang, Xiaotian, Zhou, Joey Tianyi, Quek, Tony, Poria, Soujanya, Liu, Zuozhu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PAD: Personalized Alignment of LLMs at Decoding-Time
by: Chen, Ruizhe, et al.
Published: (2024)

Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment
by: Zhang, Xiaotian, et al.
Published: (2025)

DiffPO: A causal diffusion model for learning distributions of potential outcomes
by: Ma, Yuchen, et al.
Published: (2024)

Inference Time Alignment with Reward-Guided Tree Search
by: Hung, Chia-Yu, et al.
Published: (2024)

Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic
by: Bhardwaj, Rishabh, et al.
Published: (2024)

Modest-Align: Data-Efficient Alignment for Vision-Language Models
by: Liu, Jiaxiang, et al.
Published: (2025)

Towards Robust Instruction Tuning on Multimodal Large Language Models
by: Han, Wei, et al.
Published: (2024)

Identifying and Mitigating Social Bias Knowledge in Language Models
by: Chen, Ruizhe, et al.
Published: (2024)

Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations
by: Hazra, Rima, et al.
Published: (2024)

Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
by: Majumder, Navonil, et al.
Published: (2024)

Modality-Fair Preference Optimization for Trustworthy MLLM Alignment
by: Jiang, Songtao, et al.
Published: (2024)

Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
by: Lei, Jingdi, et al.
Published: (2025)

BiasGuard: A Reasoning-enhanced Bias Detection Tool For Large Language Models
by: Fan, Zhiting, et al.
Published: (2025)

InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment
by: Lu, Yunhong, et al.
Published: (2025)

PromptDistill: Query-based Selective Token Retention in Intermediate Layers for Efficient Large Language Model Inference
by: Jin, Weisheng, et al.
Published: (2025)

AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization
by: Liu, Chaohu, et al.
Published: (2025)

ComPO: Preference Alignment via Comparison Oracles
by: Chen, Peter, et al.
Published: (2025)

Harnessing Large Language Models for Scientific Novelty Detection
by: Liu, Yan, et al.
Published: (2025)

Large Language Model Bias Mitigation from the Perspective of Knowledge Editing
by: Chen, Ruizhe, et al.
Published: (2024)

Sowing the Wind, Reaping the Whirlwind: The Impact of Editing Language Models
by: Hazra, Rima, et al.
Published: (2024)

PREMISE: Matching-based Prediction for Accurate Review Recommendation
by: Han, Wei, et al.
Published: (2025)

Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model
by: Kang, Jaeyong, et al.
Published: (2023)

Not All Votes Count! Programs as Verifiers Improve Self-Consistency of Language Models for Math Reasoning
by: Toh, Vernon Y. H., et al.
Published: (2024)

Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models
by: Han, Wei, et al.
Published: (2023)

Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense
by: Shen, Siqi, et al.
Published: (2024)

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery
by: Yang, Zonglin, et al.
Published: (2023)

AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment
by: Deng, Ruibo, et al.
Published: (2025)

Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models
by: Chia, Yew Ken, et al.
Published: (2024)

Diffusion Blend: Inference-Time Multi-Preference Alignment for Diffusion Models
by: Cheng, Min, et al.
Published: (2025)

DELLA-Merging: Reducing Interference in Model Merging through Magnitude-Based Sampling
by: Deep, Pala Tej, et al.
Published: (2024)

HyperTTS: Parameter Efficient Adaptation in Text to Speech using Hypernetworks
by: Li, Yingting, et al.
Published: (2024)

Leveraging Parameter-Efficient Transfer Learning for Multi-Lingual Text-to-Speech Adaptation
by: Li, Yingting, et al.
Published: (2024)

Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learned
by: Ong, Brandon, et al.
Published: (2025)

KnowPO: Knowledge-aware Preference Optimization for Controllable Knowledge Selection in Retrieval-Augmented Language Models
by: Zhang, Ruizhe, et al.
Published: (2024)

$δ$-mem: Efficient Online Memory for Large Language Models
by: Lei, Jingdi, et al.
Published: (2026)

Learnable Privacy Neurons Localization in Language Models
by: Chen, Ruizhe, et al.
Published: (2024)

FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering
by: Li, Yichen, et al.
Published: (2025)

Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues
by: Zhang, Xiaotian, et al.
Published: (2025)

Stacked from One: Multi-Scale Self-Injection for Context Window Extension
by: Han, Wei, et al.
Published: (2026)

Ruby Teaming: Improving Quality Diversity Search with Memory for Automated Red Teaming
by: Han, Vernon Toh Yan, et al.
Published: (2024)