Cover Image

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Jian, Huang, Haojing, Zhang, Yujia, Xu, Pengfei, Chen, Xi, Song, Rui, Shi, Lida, Wang, Jingwen, Xu, Hao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
by: Li, Jian, et al.
Published: (2025)

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
by: Yin, Yueqin, et al.
Published: (2024)

Enhancing Multimodal Large Language Models for Ancient Chinese Character Evolution Analysis via Glyph-Driven Fine-Tuning
by: Song, Rui, et al.
Published: (2026)

Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models
by: Xiang, Hao, et al.
Published: (2024)

Self-supervised Attribute-aware Dynamic Preference Ranking Alignment
by: Yang, Hongyu, et al.
Published: (2025)

Dynamic Noise Preference Optimization: Self-Improvement of Large Language Models with Self-Synthetic Data
by: Yang, Haoyan, et al.
Published: (2025)

Self-Evolutionary Large Language Models through Uncertainty-Enhanced Preference Optimization
by: Wang, Jianing, et al.
Published: (2024)

Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization
by: Pan, Yurui, et al.
Published: (2026)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
by: Cheng, Pengyu, et al.
Published: (2023)

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
by: Zhu, Wenhong, et al.
Published: (2024)

Self-Consistency Preference Optimization
by: Prasad, Archiki, et al.
Published: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
by: Chen, Qiyuan, et al.
Published: (2025)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

IPO: Your Language Model is Secretly a Preference Classifier
by: Garg, Shivank, et al.
Published: (2025)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
by: Zhou, Wenxuan, et al.
Published: (2024)

ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictions
by: Zhang, Xu, et al.
Published: (2024)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Preference Consistency Matters: Enhancing Preference Learning in Language Models with Automated Self-Curation of Training Corpora
by: Lee, JoonHo, et al.
Published: (2024)

CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferences
by: Pokharel, Rhitabrat, et al.
Published: (2025)

Beyond Under-Alignment: Atomic Preference Enhanced Factuality Tuning for Large Language Models
by: Yuan, Hongbang, et al.
Published: (2024)

TSO: Self-Training with Scaled Preference Optimization
by: Chen, Kaihui, et al.
Published: (2024)

Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models
by: Liu, Biao, et al.
Published: (2025)

ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent
by: Yin, Shangjian, et al.
Published: (2024)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
by: Wan, Xu, et al.
Published: (2025)

Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
by: Wu, Jingwen, et al.
Published: (2026)

Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback
by: Kim, Kyuyoung, et al.
Published: (2024)

Understanding Reference Policies in Direct Preference Optimization
by: Liu, Yixin, et al.
Published: (2024)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
by: Wang, Fei, et al.
Published: (2024)

Direct Judgement Preference Optimization
by: Wang, Peifeng, et al.
Published: (2024)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

VRM: Teaching Reward Models to Understand Authentic Human Preferences
by: Liu, Biao, et al.
Published: (2026)

Probability-Consistent Preference Optimization for Enhanced LLM Reasoning
by: Yang, Yunqiao, et al.
Published: (2025)

Shortcut Learning in In-Context Learning: A Survey
by: Song, Rui, et al.
Published: (2024)

Multiplayer Nash Preference Optimization
by: Wu, Fang, et al.
Published: (2025)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization
by: Pi, Renjie, et al.
Published: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)