Cover Image

MallowsPO: Fine-Tune Your LLM with Preference Dispersions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Haoxian, Zhao, Hanyang, Lam, Henry, Yao, David, Tang, Wenpin
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning
by: Zhao, Hanyang, et al.
Published: (2025)

RPO: Fine-Tuning Visual Generative Models via Rich Vision-Language Preferences
by: Zhao, Hanyang, et al.
Published: (2025)

Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning
by: Zhao, Hanyang, et al.
Published: (2024)

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
by: Sheng, Jiayuan, et al.
Published: (2025)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
by: Zhao, Hanyang, et al.
Published: (2025)

Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey
by: Winata, Genta Indra, et al.
Published: (2024)

SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations
by: Zhang, Haoting, et al.
Published: (2025)

RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization
by: Zhao, Hanyang, et al.
Published: (2024)

Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
by: Guo, Zhengyi, et al.
Published: (2025)

Contractive Diffusion Probabilistic Models
by: Tang, Wenpin, et al.
Published: (2024)

Alignment Dynamics in LLM Fine-Tuning
by: Huang, Yuhan, et al.
Published: (2026)

Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial
by: Tang, Wenpin, et al.
Published: (2024)

CurvZO: Adaptive Curvature-Guided Sparse Zeroth-Order Optimization for Efficient LLM Fine-Tuning
by: Wang, Shuo, et al.
Published: (2026)

DipLLM: Fine-Tuning LLM for Strategic Decision-making in Diplomacy
by: Xu, Kaixuan, et al.
Published: (2025)

Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning
by: Qin, Kai, et al.
Published: (2025)

RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation
by: Liu, Jun, et al.
Published: (2025)

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
by: Liu, Siyuan, et al.
Published: (2026)

Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection
by: Lai, Zhixin, et al.
Published: (2024)

PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
by: Verma, Richa, et al.
Published: (2026)

ComPO: Preference Alignment via Comparison Oracles
by: Chen, Peter, et al.
Published: (2025)

Internalizing Curriculum Judgment for LLM Reinforcement Fine-Tuning
by: Zheng, Han, et al.
Published: (2026)

Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
by: Schweighofer, Kajetan, et al.
Published: (2026)

LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection
by: Zeng, Xinyue, et al.
Published: (2025)

Goal-Conditioned Supervised Learning for LLM Fine-Tuning
by: Li, Shijun, et al.
Published: (2026)

Implicit Federated In-context Learning For Task-Specific LLM Fine-Tuning
by: Li, Dongcheng, et al.
Published: (2025)

Filter-then-Weight: Online Data Selection and Reweighting for LLM Fine-Tuning
by: Wang, Fangxin, et al.
Published: (2026)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models
by: Liu, Qi, et al.
Published: (2025)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs
by: Kachroo, Darsh, et al.
Published: (2026)

SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
by: Hossain, Saad, et al.
Published: (2025)

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
by: Chen, Yupeng, et al.
Published: (2024)

A Layer-wise Analysis of Supervised Fine-Tuning
by: Zhao, Qinghua, et al.
Published: (2026)

FairPO: Robust Preference Optimization for Fair Multi-Label Learning
by: Mondal, Soumen Kumar, et al.
Published: (2025)

Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
by: Tang, Zhenchao, et al.
Published: (2025)

Understanding the Performance and Estimating the Cost of LLM Fine-Tuning
by: Xia, Yuchen, et al.
Published: (2024)

Representation Without Reward: A JEPA Audit for LLM Fine-Tuning
by: Sengupta, Biswa
Published: (2026)

FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents
by: Li, Qizheng, et al.
Published: (2026)

SplitFrozen: Split Learning with Device-side Model Frozen for Fine-Tuning LLM on Heterogeneous Resource-Constrained Devices
by: Ma, Jian, et al.
Published: (2025)