Cover Image

APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Srewa, Mahmoud, Zhao, Tianyu, Elmalaki, Salma
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs
by: Srewa, Mahmoud, et al.
Published: (2025)

PluralLLM: Pluralistic Alignment in LLMs via Federated Learning
by: Srewa, Mahmoud, et al.
Published: (2025)

FinP: Fairness-in-Privacy in Federated Learning by Addressing Disparities in Privacy Risk
by: Zhao, Tianyu, et al.
Published: (2025)

Adaptive Alignment: Dynamic Preference Adjustments via Multi-Objective Reinforcement Learning for Pluralistic AI
by: Harland, Hadassah, et al.
Published: (2024)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
by: Imai, Saki, et al.
Published: (2026)

Adaptive Margin RLHF via Preference over Preferences
by: Chittepu, Yaswanth, et al.
Published: (2025)

Pairwise Calibrated Rewards for Pluralistic Alignment
by: Halpern, Daniel, et al.
Published: (2025)

Democratic Preference Alignment via Sortition-Weighted RLHF
by: Sana, Suvadip, et al.
Published: (2026)

MaxMin-RLHF: Alignment with Diverse Human Preferences
by: Chakraborty, Souradip, et al.
Published: (2024)

Policy-labeled Preference Learning: Is Preference Enough for RLHF?
by: Cho, Taehyun, et al.
Published: (2025)

Pluralistic Alignment Over Time
by: Klassen, Toryn Q., et al.
Published: (2024)

Multi-objective Reinforcement Learning: A Tool for Pluralistic Alignment
by: Vamplew, Peter, et al.
Published: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
by: Zhou, Wenxuan, et al.
Published: (2024)

Towards Reliable Alignment: Uncertainty-aware RLHF
by: Banerjee, Debangshu, et al.
Published: (2024)

RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs
by: Dang, John, et al.
Published: (2024)

GEM: Generative Entropy-Guided Preference Modeling for Few-shot Alignment of LLMs
by: Zhao, Yiyang, et al.
Published: (2025)

Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models
by: Guo, Hanze, et al.
Published: (2025)

RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation
by: Park, Chanwoo, et al.
Published: (2024)

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
by: Belakaria, Syrine, et al.
Published: (2025)

Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
by: Siththaranjan, Anand, et al.
Published: (2023)

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
by: Liang, Ren-Wei, et al.
Published: (2025)

Towards Reward Fairness in RLHF: From a Resource Allocation Perspective
by: Ouyang, Sheng, et al.
Published: (2025)

Pluralistic Alignment for Healthcare: A Role-Driven Framework
by: Zhong, Jiayou, et al.
Published: (2025)

Active Preference Optimization for Sample Efficient RLHF
by: Das, Nirjhar, et al.
Published: (2024)

Reward-Robust RLHF in LLMs
by: Yan, Yuzi, et al.
Published: (2024)

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
by: Cen, Shicong, et al.
Published: (2024)

VISPA: Pluralistic Alignment via Automatic Value Selection and Activation
by: Zheng, Shenyan, et al.
Published: (2026)

VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare
by: Shetty, Anudeex, et al.
Published: (2025)

Beyond RLHF and NLHF: Population-Proportional Alignment under an Axiomatic Framework
by: Kim, Kihyun, et al.
Published: (2025)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
by: Zhang, Shenao, et al.
Published: (2024)

Sample Efficient Preference Alignment in LLMs via Active Exploration
by: Mehta, Viraj, et al.
Published: (2023)

Influencing Humans to Conform to Preference Models for RLHF
by: Hatgis-Kessell, Stephane, et al.
Published: (2025)

Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences
by: Falahati, Ali, et al.
Published: (2026)

Avoiding $\mathbf{exp(R_{max})}$ scaling in RLHF through Preference-based Exploration
by: Chen, Mingyu, et al.
Published: (2025)

Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
by: Kumarappan, Adarsh, et al.
Published: (2026)

Conditional Equivalence of DPO and RLHF: Implicit Assumption, Failure Modes, and Provable Alignment
by: Yang, Zhiqin, et al.
Published: (2026)

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
by: Xiao, Youshao, et al.
Published: (2023)

A Persona-Based Evaluation Framework for Pluralistic Alignment in Generative AI
by: Karagoz, Atahan
Published: (2026)

Being Considerate as a Pathway Towards Pluralistic Alignment for Agentic AI
by: Alamdari, Parand A., et al.
Published: (2024)

Aligning CodeLLMs with Direct Preference Optimization
by: Miao, Yibo, et al.
Published: (2024)