Cover Image

Aligning LLMs by Predicting Preferences from User Writing Samples

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Aroca-Ouellette, Stéphane, Mackraz, Natalie, Theobald, Barry-John, Metcalf, Katherine
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories
by: Aroca-Ouellette, Stephane, et al.
Published: (2024)

Sample-Efficient Preference-based Reinforcement Learning with Dynamics Aware Rewards
by: Metcalf, Katherine, et al.
Published: (2024)

Theoretical Limits of Language Model Alignment
by: Paes, Lucas Monteiro, et al.
Published: (2026)

Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
by: Wang, Yinong Oliver, et al.
Published: (2025)

Bias after Prompting: Persistent Discrimination in Large Language Models
by: Sivakumar, Nivedha, et al.
Published: (2025)

Evaluating Gender Bias Transfer between Pre-trained and Prompt-Adapted Language Models
by: Mackraz, Natalie, et al.
Published: (2024)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

Steering into New Embedding Spaces: Analyzing Cross-Lingual Alignment Induced by Model Interventions in Multilingual Language Models
by: Sundar, Anirudh, et al.
Published: (2025)

Implicitly Aligning Humans and Autonomous Agents through Shared Task Abstractions
by: Aroca-Ouellette, Stéphane, et al.
Published: (2025)

Large Language Models as Generalizable Policies for Embodied Tasks
by: Szot, Andrew, et al.
Published: (2023)

On the Way to LLM Personalization: Learning to Remember User Conversations
by: Magister, Lucie Charlotte, et al.
Published: (2024)

Towards Efficient Neurally-Guided Program Induction for ARC-AGI
by: Ouellette, Simon
Published: (2024)

Aligning LLM Agents by Learning Latent Preference from User Edits
by: Gao, Ge, et al.
Published: (2024)

Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
by: Ahmed, Kareem, et al.
Published: (2026)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs
by: Zhao, Siyan, et al.
Published: (2025)

Aligning (Medical) LLMs for (Counterfactual) Fairness
by: Poulain, Raphael, et al.
Published: (2024)

Hindsight PRIORs for Reward Learning from Human Preferences
by: Verma, Mudit, et al.
Published: (2024)

Principled Fine-tuning of LLMs from User-Edits: A Medley of Preference, Supervision, and Reward
by: Misra, Dipendra, et al.
Published: (2026)

DSO: Direct Steering Optimization for Bias Mitigation
by: Paes, Lucas Monteiro, et al.
Published: (2025)

COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences
by: Liu, Yixin, et al.
Published: (2024)

Preference Optimization with Multi-Sample Comparisons
by: Wang, Chaoqi, et al.
Published: (2024)

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
by: Bansal, Hritik, et al.
Published: (2023)

Aligning Language Models from User Interactions
by: Buening, Thomas Kleine, et al.
Published: (2026)

MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples
by: Xie, Shuo, et al.
Published: (2024)

Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
by: Qiu, Wenjie, et al.
Published: (2025)

HinTel-AlignBench: A Framework and Benchmark for Hindi-Telugu with English-Aligned Samples
by: Chigrupaatii, Rishikant, et al.
Published: (2025)

RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs
by: Wu, Jiaxing, et al.
Published: (2024)

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization
by: Zhou, Zhenglin, et al.
Published: (2025)

ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMs
by: Zabolotnyi, Artem, et al.
Published: (2025)

Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning
by: Zhang, Yuheng, et al.
Published: (2024)

ExpertLens: Activation steering features are highly interpretable
by: Fedzechkina, Masha, et al.
Published: (2025)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
by: Zhang, Xuan, et al.
Published: (2024)

Diversity Augmentation of Dynamic User Preference Data for Boosting Personalized Text Summarizers
by: Chatterjee, Parthiv, et al.
Published: (2025)

Aligning the True Semantics: Constrained Decoupling and Distribution Sampling for Cross-Modal Alignment
by: Ma, Xiang, et al.
Published: (2026)

Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs
by: Gallego, Víctor
Published: (2024)

Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?
by: Tang, Zilu, et al.
Published: (2025)

The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm
by: Aakanksha, et al.
Published: (2024)

Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation
by: Bai, Fengshuo, et al.
Published: (2024)

BOND: Aligning LLMs with Best-of-N Distillation
by: Sessa, Pier Giuseppe, et al.
Published: (2024)