Cover Image

FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhu, Mingye, Liu, Yi, Wang, Quan, Guo, Junbo, Mao, Zhendong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On-the-fly Preference Alignment via Principle-Guided Decoding
by: Zhu, Mingye, et al.
Published: (2025)

Leveraging Importance Sampling to Detach Alignment Modules from Large Language Models
by: Liu, Yi, et al.
Published: (2025)

Alignment-Enhanced Decoding:Defending via Token-Level Adaptive Refining of Probability Distributions
by: Liu, Quan, et al.
Published: (2024)

Leveraging Robust Optimization for LLM Alignment under Distribution Shifts
by: Zhu, Mingye, et al.
Published: (2025)

LIRE: listwise reward enhancement for preference alignment
by: Zhu, Mingye, et al.
Published: (2024)

Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
by: Yin, Yueqin, et al.
Published: (2024)

SparseRM: A Lightweight Preference Modeling with Sparse Autoencoder
by: Liu, Dengcan, et al.
Published: (2025)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization
by: Wang, Yilong, et al.
Published: (2026)

Defending LLMs against Jailbreaking Attacks via Backtranslation
by: Wang, Yihan, et al.
Published: (2024)

Constrain Alignment with Sparse Autoencoders
by: Yin, Qingyu, et al.
Published: (2024)

Enhancing Persona Following at Decoding Time via Dynamic Importance Estimation for Role-Playing Agents
by: Liu, Yuxin, et al.
Published: (2026)

PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
by: Wu, Jiayi, et al.
Published: (2024)

ICDPO: Effectively Borrowing Alignment Capability of Others via In-context Direct Preference Optimization
by: Song, Feifan, et al.
Published: (2024)

Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization
by: Dong, Zhijin
Published: (2025)

Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment
by: Guo, Yiju, et al.
Published: (2024)

ELDER: Enhancing Lifelong Model Editing with Mixture-of-LoRA
by: Li, Jiaang, et al.
Published: (2024)

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization
by: Lyu, Yougang, et al.
Published: (2024)

Prompt Optimization Is a Coin Flip: Diagnosing When It Helps in Compound AI Systems
by: Zhang, Xing, et al.
Published: (2026)

ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
by: Xu, Benfeng, et al.
Published: (2023)

Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach
by: Li, Ruizhe, et al.
Published: (2025)

Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment
by: Zhang, Jianfei, et al.
Published: (2024)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
by: Wang, Jialu, et al.
Published: (2026)

Multi-Faceted Self-Consistent Preference Alignment for Query Rewriting in Conversational Search
by: Cao, Zhiyu, et al.
Published: (2026)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

Is On-Policy Data always the Best Choice for Direct Preference Optimization-based LM Alignment?
by: Sun, Zetian, et al.
Published: (2025)

DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoning
by: Wang, Yaxuan, et al.
Published: (2025)

Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
by: Yi, Jingwei, et al.
Published: (2023)

Word Alignment as Preference for Machine Translation
by: Wu, Qiyu, et al.
Published: (2024)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction
by: Jiang, Tanqiu, et al.
Published: (2024)

Stepwise Alignment for Constrained Language Model Policy Optimization
by: Wachi, Akifumi, et al.
Published: (2024)

Preference-Aware Memory Update for Long-Term LLM Agents
by: Sun, Haoran, et al.
Published: (2025)

TODO: Enhancing LLM Alignment with Ternary Preferences
by: Guo, Yuxiang, et al.
Published: (2024)

MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Tools
by: Guo, Zikang, et al.
Published: (2025)

Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
by: Song, Feifan, et al.
Published: (2025)

Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment
by: D'Oosterlinck, Karel, et al.
Published: (2024)

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment
by: Yang, Wen, et al.
Published: (2025)