Imagen de Portada

Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Mingzhi, Ma, Chengdong, Chen, Qizhi, Meng, Linjian, Han, Yang, Xiao, Jiancong, Zhang, Zhaowei, Huo, Jing, Su, Weijie J., Yang, Yaodong
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMs
por: Zhang, Zhaowei, et al.
Publicado: (2025)

Roadmap on Incentive Compatibility for AI Alignment and Governance in Sociotechnical Systems
por: Zhang, Zhaowei, et al.
Publicado: (2024)

Efficient Last-iterate Convergence Algorithms in Solving Games
por: Meng, Linjian, et al.
Publicado: (2023)

Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching
por: Shi, Zhekun, et al.
Publicado: (2025)

Panacea: Pareto Alignment via Preference Adaptation for LLMs
por: Zhong, Yifan, et al.
Publicado: (2024)

Efficient Model-agnostic Alignment via Bayesian Persuasion
por: Bai, Fengshuo, et al.
Publicado: (2024)

Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibrium
por: Liu, Kaizhao, et al.
Publicado: (2025)

Last-iterate Convergence in Regularized Graphon Mean Field Game
por: Dong, Jing, et al.
Publicado: (2024)

On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularization
por: Xiao, Jiancong, et al.
Publicado: (2024)

ValueDCG: Measuring Comprehensive Human Value Understanding Ability of Language Models
por: Zhang, Zhaowei, et al.
Publicado: (2023)

Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization
por: Xiao, Jiancong, et al.
Publicado: (2024)

Heterogeneous Value Alignment Evaluation for Large Language Models
por: Zhang, Zhaowei, et al.
Publicado: (2023)

Last-iterate Convergence Separation between Extra-gradient and Optimism in Constrained Periodic Games
por: Feng, Yi, et al.
Publicado: (2024)

Evolving Diverse Red-team Language Models in Multi-round Multi-agent Games
por: Ma, Chengdong, et al.
Publicado: (2023)

Fine-Tuning Attention Modules Only: Enhancing Weight Disentanglement in Task Arithmetic
por: Jin, Ruochen, et al.
Publicado: (2024)

Optimal Primal-Dual Algorithm with Last iterate Convergence Guarantees for Stochastic Convex Optimization Problems
por: Boob, Digvijay, et al.
Publicado: (2024)

PoliCon: Evaluating LLMs on Achieving Diverse Political Consensus Objectives
por: Zhang, Zhaowei, et al.
Publicado: (2025)

Differentially Private Preference Data Synthesis for Large Language Model Alignment
por: Gao, Fengyu, et al.
Publicado: (2026)

Finding Kissing Numbers with Game-theoretic Reinforcement Learning
por: Ma, Chengdong, et al.
Publicado: (2025)

LLMs Know More Than Words: A Genre Study with Syntax, Metaphor & Phonetics
por: Shi, Weiye, et al.
Publicado: (2025)

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning
por: Chen, Sirui, et al.
Publicado: (2023)

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory
por: Xiao, Jiancong, et al.
Publicado: (2025)

Falcon: Fast Visuomotor Policies via Partial Denoising
por: Chen, Haojun, et al.
Publicado: (2025)

Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?
por: Gölz, Paul, et al.
Publicado: (2025)

Last-iterate Convergence of ADMM on Multi-affine Quadratic Equality Constrained Problem
por: Chao, Yutong, et al.
Publicado: (2026)

Self-Play Preference Optimization for Language Model Alignment
por: Wu, Yue, et al.
Publicado: (2024)

Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning
por: Fan, Wenzhe, et al.
Publicado: (2024)

Goal Discovery with Causal Capacity for Efficient Reinforcement Learning
por: Yu, Yan, et al.
Publicado: (2025)

Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach
por: Xiao, Jiancong, et al.
Publicado: (2025)

Conflux-PSRO: Effectively Leveraging Collective Advantages in Policy Space Response Oracles
por: Huang, Yucong, et al.
Publicado: (2024)

Benchmarking Multi-National Value Alignment for Large Language Models
por: Shi, Weijie, et al.
Publicado: (2025)

Triple Preference Optimization: Achieving Better Alignment using a Single Step Optimization
por: Saeidi, Amir, et al.
Publicado: (2024)

Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization
por: Wang, Yilong, et al.
Publicado: (2026)

Sample-Efficient Regret-Minimizing Double Oracle in Extensive-Form Games
por: Tang, Xiaohang, et al.
Publicado: (2024)

Instant Preference Alignment for Text-to-Image Diffusion Models
por: Li, Yang, et al.
Publicado: (2025)

Truthful Two-Obnoxious-Facility Location Games with Optional Preferences and Minimum Distance Constraint
por: Han, Xiaojia, et al.
Publicado: (2025)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
por: Zhou, Zhanhui, et al.
Publicado: (2023)

SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset
por: Dai, Josef, et al.
Publicado: (2024)

Last-Iterate Convergence of Anchored Gradient Descent
por: Cai, Yang, et al.
Publicado: (2026)

Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback
por: Zhou, Runlong, et al.
Publicado: (2025)