Cover Image

Direct Alignment with Heterogeneous Preferences

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shirali, Ali, Nasr-Esfahany, Arash, Alomar, Abdullah, Mirtaheri, Parsa, Abebe, Rediet, Procaccia, Ariel
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Hidden Cost of Waiting for Accurate Predictions
by: Shirali, Ali, et al.
Published: (2025)

The Burden of Interactive Alignment with Inconsistent Preferences
by: Shirali, Ali
Published: (2025)

Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing
by: Mirtaheri, Parsa, et al.
Published: (2026)

Online Reinforcement Learning in Non-Stationary Context-Driven Environments
by: Hamadanian, Pouya, et al.
Published: (2023)

Allocation Requires Prediction Only if Inequality Is Low
by: Shirali, Ali, et al.
Published: (2024)

Pairwise Calibrated Rewards for Pluralistic Alignment
by: Halpern, Daniel, et al.
Published: (2025)

Clone-Robust AI Alignment
by: Procaccia, Ariel D., et al.
Published: (2025)

Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones
by: Mirtaheri, Parsa, et al.
Published: (2025)

Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Care
by: Shirali, Ali, et al.
Published: (2023)

GRASP: Deterministic argument ranking in interaction graphs
by: Misra, Diganta, et al.
Published: (2026)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization
by: Zhou, Zhanhui, et al.
Published: (2023)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

ActiveDPO: Active Direct Preference Optimization for Sample-Efficient Alignment
by: Lin, Xiaoqiang, et al.
Published: (2025)

Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
by: Xu, Zaiyan, et al.
Published: (2025)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Generative Social Choice: The Next Generation
by: Boehmer, Niclas, et al.
Published: (2025)

Axioms for AI Alignment from Human Feedback
by: Ge, Luise, et al.
Published: (2024)

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
by: Wang, Jialu, et al.
Published: (2026)

The Sign Estimator: LLM Alignment in the Face of Choice Heterogeneity
by: Aouad, Ali, et al.
Published: (2025)

Continuous-Utility Direct Preference Optimization
by: Mohsin, Muhammad Ahmed, et al.
Published: (2026)

Alternates, Assemble! Selecting Optimal Alternates for Citizens' Assemblies
by: Assos, Angelos, et al.
Published: (2025)

Policy Aggregation
by: Alamdari, Parand A., et al.
Published: (2024)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
by: Aakanksha, et al.
Published: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Lawma: The Power of Specialization for Legal Annotation
by: Dominguez-Olmedo, Ricardo, et al.
Published: (2024)

Self-Improving Robust Preference Optimization
by: Choi, Eugene, et al.
Published: (2024)

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
by: Liang, Ren-Wei, et al.
Published: (2025)

Adversarial Preference Learning for Robust LLM Alignment
by: Wang, Yuanfu, et al.
Published: (2025)

Preference-Based Alignment of Discrete Diffusion Models
by: Borso, Umberto, et al.
Published: (2025)

ADPO: Anchored Direct Preference Optimization
by: Zixian, Wang
Published: (2025)

Embeddings for Preferences, Not Semantics
by: Blair, Carter, et al.
Published: (2026)

Strategic Classification With Externalities
by: Hossain, Safwan, et al.
Published: (2024)

Towards Improving Long-Tail Entity Predictions in Temporal Knowledge Graphs through Global Similarity and Weighted Sampling
by: Mirtaheri, Mehrnoosh, et al.
Published: (2025)

Preference Learning for AI Alignment: a Causal Perspective
by: Kobalczyk, Katarzyna, et al.
Published: (2025)

PIPA: Preference Alignment as Prior-Informed Statistical Estimation
by: Li, Junbo, et al.
Published: (2025)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)

From Coefficients to Directions: Rethinking Model Merging with Directional Alignment
by: Chen, Zhikang, et al.
Published: (2025)

RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation
by: Park, Chanwoo, et al.
Published: (2024)

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
by: Zeng, Siliang, et al.
Published: (2025)