Image de couverture de livre

SimPO: Simple Preference Optimization with a Reference-Free Reward

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Meng, Yu, Xia, Mengzhou, Chen, Danqi
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
par: Zhong, Zexuan, et autres
Publié: (2024)

The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
par: Zhu, Xinyu, et autres
Publié: (2025)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
par: Xia, Mengzhou, et autres
Publié: (2023)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
par: Zhu, Zining, et autres
Publié: (2025)

LESS: Selecting Influential Data for Targeted Instruction Tuning
par: Xia, Mengzhou, et autres
Publié: (2024)

References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulation
par: Kim, Doyoung, et autres
Publié: (2025)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
par: Razin, Noam, et autres
Publié: (2024)

$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization
par: Han, Jiaqi, et autres
Publié: (2024)

LiPO: Listwise Preference Optimization through Learning-to-Rank
par: Liu, Tianqi, et autres
Publié: (2024)

T-REG: Preference Optimization with Token-Level Reward Regularization
par: Zhou, Wenxuan, et autres
Publié: (2024)

DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization
par: She, Shuaijie, et autres
Publié: (2025)

Detecting Pretraining Data from Large Language Models
par: Shi, Weijia, et autres
Publié: (2023)

LitSearch: A Retrieval Benchmark for Scientific Literature Search
par: Ajith, Anirudh, et autres
Publié: (2024)

Group Robust Preference Optimization in Reward-free RLHF
par: Ramesh, Shyam Sundhar, et autres
Publié: (2024)

Robust Preference Optimization through Reward Model Distillation
par: Fisch, Adam, et autres
Publié: (2024)

Understanding Reference Policies in Direct Preference Optimization
par: Liu, Yixin, et autres
Publié: (2024)

RankPO: Preference Optimization for Job-Talent Matching
par: Zhang, Yafei, et autres
Publié: (2025)

Multi-Reference Preference Optimization for Large Language Models
par: Le, Hung, et autres
Publié: (2024)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
par: Chen, Guanzheng, et autres
Publié: (2025)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
par: Lin, Yong, et autres
Publié: (2024)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
par: Zhu, Mingkang, et autres
Publié: (2025)

Generalizing Reward Modeling for Out-of-Distribution Preference Learning
par: Jia, Chen
Publié: (2024)

Evaluating Large Language Models at Evaluating Instruction Following
par: Zeng, Zhiyuan, et autres
Publié: (2023)

Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
par: Li, Wenzhe, et autres
Publié: (2025)

Trainable Transformer in Transformer
par: Panigrahi, Abhishek, et autres
Publié: (2023)

ComPO: Preference Alignment via Comparison Oracles
par: Chen, Peter, et autres
Publié: (2025)

SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters
par: Xiao, Teng, et autres
Publié: (2025)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
par: Sun, Shengyang, et autres
Publié: (2025)

Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences
par: Karlekar, Sweta, et autres
Publié: (2026)

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
par: Bhaskar, Adithya, et autres
Publié: (2024)

MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
par: Zhu, Kangyu, et autres
Publié: (2024)

SPARK: Stepwise Process-Aware Rewards for Reference-Free Reinforcement Learning
par: Rahman, Salman, et autres
Publié: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
par: Wu, Junkang, et autres
Publié: (2024)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
par: Yoon, Hee Suk, et autres
Publié: (2025)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
par: Peng, Andi, et autres
Publié: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
par: Rafailov, Rafael, et autres
Publié: (2023)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
par: Qiu, Wenjie, et autres
Publié: (2025)

Selective Preference Optimization via Token-Level Reward Function Estimation
par: Yang, Kailai, et autres
Publié: (2024)

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
par: Chen, Howard, et autres
Publié: (2025)

What is in Your Safe Data? Identifying Benign Data that Breaks Safety
par: He, Luxi, et autres
Publié: (2024)