Cover Image

Sequence-level Large Language Model Training with Contrastive Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Feng, Zhili, Ram, Dhananjay, Hawkins, Cole, Rawal, Aditya, Zhao, Jinman, Zha, Sheng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DEM: Distribution Edited Model for Training with Mixed Data Distributions
by: Ram, Dhananjay, et al.
Published: (2024)

Group Preference Optimization: Few-Shot Alignment of Large Language Models
by: Zhao, Siyan, et al.
Published: (2023)

DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning
by: Karaman, Batuhan K., et al.
Published: (2026)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
by: Bansal, Hritik, et al.
Published: (2023)

Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
by: Bansal, Hritik, et al.
Published: (2024)

Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
by: Liu, Xiaoze, et al.
Published: (2026)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

Language Models Can Predict Their Own Behavior
by: Ashok, Dhananjay, et al.
Published: (2025)

Probing the Decision Boundaries of In-context Learning in Large Language Models
by: Zhao, Siyan, et al.
Published: (2024)

Large Language Models for Cross-lingual Emotion Detection
by: Kadiyala, Ram Mohan Rao
Published: (2024)

Mitigating Hallucinated Translations in Large Language Models with Hallucination-focused Preference Optimization
by: Tang, Zilu, et al.
Published: (2025)

Existing Large Language Model Unlearning Evaluations Are Inconclusive
by: Feng, Zhili, et al.
Published: (2025)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
by: Jia, Sheng, et al.
Published: (2025)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Adapting Language Models via Token Translation
by: Feng, Zhili, et al.
Published: (2024)

BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
by: Lee, Gihun, et al.
Published: (2024)

Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
by: D'Oosterlinck, Karel, et al.
Published: (2024)

Active Preference Learning for Large Language Models
by: Muldrew, William, et al.
Published: (2024)

TSO: Self-Training with Scaled Preference Optimization
by: Chen, Kaihui, et al.
Published: (2024)

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
by: Li, Junsong, et al.
Published: (2025)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
by: Zhu, Mingkang, et al.
Published: (2025)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

Optimizing Large Language Models for Turkish: New Methodologies in Corpus Selection and Training
by: Kesgin, H. Toprak, et al.
Published: (2024)

CELL your Model: Contrastive Explanations for Large Language Models
by: Luss, Ronny, et al.
Published: (2024)

Param$Δ$ for Direct Weight Mixing: Post-Train Large Language Model at Zero Cost
by: Cao, Sheng, et al.
Published: (2025)

Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models
by: Zhao, Siyan, et al.
Published: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
by: Zhang, Wenxuan, et al.
Published: (2024)

Improving Large Language Model Safety with Contrastive Representation Learning
by: Simko, Samuel, et al.
Published: (2025)

AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models
by: Beux, Yann Le, et al.
Published: (2025)

Bayesian Network Fusion of Large Language Models for Sentiment Analysis
by: Amirzadeh, Rasoul, et al.
Published: (2025)

Large Language Models as Optimizers
by: Yang, Chengrun, et al.
Published: (2023)

Sliding Window Attention Training for Efficient Large Language Models
by: Fu, Zichuan, et al.
Published: (2025)

Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
by: Rosset, Corby, et al.
Published: (2024)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Training Optimal Large Diffusion Language Models
by: Ni, Jinjie, et al.
Published: (2025)