Imagen de Portada

Preference Learning Algorithms Do Not Learn Preference Rankings

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Chen, Angelica, Malladi, Sadhika, Zhang, Lily H., Chen, Xinyi, Zhang, Qiuyi, Ranganath, Rajesh, Cho, Kyunghyun
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
por: Razin, Noam, et al.
Publicado: (2024)

Preference learning made easy: Everything should be understood through win rate
por: Zhang, Lily H., et al.
Publicado: (2025)

LESS: Selecting Influential Data for Targeted Instruction Tuning
por: Xia, Mengzhou, et al.
Publicado: (2024)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
por: Pattnaik, Pulkit, et al.
Publicado: (2024)

Active Preference Learning for Large Language Models
por: Muldrew, William, et al.
Publicado: (2024)

Preference Poisoning Attacks on Reward Model Learning
por: Wu, Junlin, et al.
Publicado: (2024)

Towards Minimal Targeted Updates of Language Models with Targeted Negative Training
por: Zhang, Lily H., et al.
Publicado: (2024)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
por: Devine, Peter
Publicado: (2024)

RankPO: Preference Optimization for Job-Talent Matching
por: Zhang, Yafei, et al.
Publicado: (2025)

Contrasting with Symile: Simple Model-Agnostic Representation Learning for Unlimited Modalities
por: Saporta, Adriel, et al.
Publicado: (2024)

Efficient semantic uncertainty quantification in language models via diversity-steered sampling
por: Park, Ji Won, et al.
Publicado: (2025)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
por: Gupta, Taneesh, et al.
Publicado: (2025)

VPO: Leveraging the Number of Votes in Preference Optimization
por: Cho, Jae Hyeon, et al.
Publicado: (2024)

Bridging the Gap Between Preference Alignment and Machine Unlearning
por: Feng, Xiaohua, et al.
Publicado: (2025)

Holistic Utility Preference Learning for Listwise Alignment
por: Zhou, Jiacong, et al.
Publicado: (2024)

Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation
por: Bai, Fengshuo, et al.
Publicado: (2024)

The Coverage Principle: How Pre-Training Enables Post-Training
por: Chen, Fan, et al.
Publicado: (2025)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

TSO: Self-Training with Scaled Preference Optimization
por: Chen, Kaihui, et al.
Publicado: (2024)

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference
por: Gao, Mingqi, et al.
Publicado: (2024)

CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning
por: Dragomir, Alexandra, et al.
Publicado: (2026)

RouteLLM: Learning to Route LLMs with Preference Data
por: Ong, Isaac, et al.
Publicado: (2024)

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
por: Hu, Michael Y., et al.
Publicado: (2026)

Anyprefer: An Agentic Framework for Preference Data Synthesis
por: Zhou, Yiyang, et al.
Publicado: (2025)

ComPO: Preference Alignment via Comparison Oracles
por: Chen, Peter, et al.
Publicado: (2025)

Geometric-Averaged Preference Optimization for Soft Preference Labels
por: Furuta, Hiroki, et al.
Publicado: (2024)

HelpSteer2-Preference: Complementing Ratings with Preferences
por: Wang, Zhilin, et al.
Publicado: (2024)

Adaptive Margin RLHF via Preference over Preferences
por: Chittepu, Yaswanth, et al.
Publicado: (2025)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
por: Wang, Haoxiang, et al.
Publicado: (2024)

Training Language Models with Language Feedback at Scale
por: Scheurer, Jérémy, et al.
Publicado: (2023)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
por: Zhang, Wenxuan, et al.
Publicado: (2024)

Approximating Human Preferences Using a Multi-Judge Learned System
por: Sprejer, Eitán, et al.
Publicado: (2025)

Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning
por: Poddar, Sriyash, et al.
Publicado: (2024)

SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
por: Kundurthy, Srivatsa, et al.
Publicado: (2026)

Orthogonal Finetuning for Direct Preference Optimization
por: Yang, Chenxu, et al.
Publicado: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
por: Kim, Dongyoung, et al.
Publicado: (2024)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
por: Jian, Chengtao, et al.
Publicado: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
por: Ye, Ziyi, et al.
Publicado: (2024)

MAPLE: A Framework for Active Preference Learning Guided by Large Language Models
por: Mahmud, Saaduddin, et al.
Publicado: (2024)

ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
por: Melikidze, Davit, et al.
Publicado: (2026)