Cover Image

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gu, Yanggan, Li, Junzhuo, Huang, Sirui, Zou, Xin, Li, Zhenghua, Hu, Xuming
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning Factual Knowledge in Structured Data with Large Language Models
by: Huang, Sirui, et al.
Published: (2024)

Aligning Large Language Models with Searcher Preferences
by: Wu, Wei, et al.
Published: (2026)

Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptation
by: Zheng, Kening, et al.
Published: (2026)

Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation
by: Li, Junzhuo, et al.
Published: (2025)

PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
by: Zhang, Rongzhi, et al.
Published: (2024)

Safer-Instruct: Aligning Language Models with Automated Preference Data
by: Shi, Taiwei, et al.
Published: (2023)

Aligning Large Language Model Behavior with Human Citation Preferences
by: Ando, Kenichiro, et al.
Published: (2026)

Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?
by: Tang, Zilu, et al.
Published: (2025)

Can LLMs Capture Human Preferences?
by: Goli, Ali, et al.
Published: (2023)

LifeAlign: Lifelong Alignment for Large Language Models with Memory-Augmented Focalized Preference Optimization
by: Li, Junsong, et al.
Published: (2025)

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness
by: Li, Jian, et al.
Published: (2024)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
by: Zhang, Xuemiao, et al.
Published: (2025)

Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations
by: Zhu, Jian-Qiao, et al.
Published: (2025)

Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models
by: Wang, Bo, et al.
Published: (2026)

TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees
by: Liao, Weibin, et al.
Published: (2024)

Preference Packing: Efficient Preference Optimization for Large Language Models
by: Cho, Jaekyung
Published: (2026)

Debate, Reflect, and Distill: Multi-Agent Feedback with Tree-Structured Preference Optimization for Efficient Language Model Enhancement
by: Zhou, Xiaofeng, et al.
Published: (2025)

Transitivity Meets Cyclicity: Explicit Preference Decomposition for Dynamic Large Language Model Alignment
by: Huang, Yucong, et al.
Published: (2026)

Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios
by: Dang, Yunkai, et al.
Published: (2024)

Improving Attributed Text Generation of Large Language Models via Preference Learning
by: Li, Dongfang, et al.
Published: (2024)

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
by: Padmakumar, Vishakh, et al.
Published: (2024)

Icon$^{2}$: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation
by: Chen, Qiyuan, et al.
Published: (2025)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
by: Yeh, Samuel, et al.
Published: (2025)

Improving Contextual ASR via Multi-grained Fusion with Large Language Models
by: Zhou, Shilin, et al.
Published: (2025)

Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis
by: Li, Junzhuo, et al.
Published: (2025)

RPO-RAG: Aligning Small LLMs with Relation-aware Preference Optimization for Knowledge Graph Question Answering
by: Um, Kaehyun, et al.
Published: (2026)

Accelerated Preference Optimization for Large Language Model Alignment
by: He, Jiafan, et al.
Published: (2024)

HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge
by: Huang, Sirui, et al.
Published: (2025)

MiniLLM: On-Policy Distillation of Large Language Models
by: Gu, Yuxian, et al.
Published: (2023)

Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
by: Liu, Yinhong, et al.
Published: (2024)

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
by: Bansal, Hritik, et al.
Published: (2023)

Self-Boosting Large Language Models with Synthetic Preference Data
by: Dong, Qingxiu, et al.
Published: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)

Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models
by: Liu, Yinhong, et al.
Published: (2024)

Aligning Model Evaluations with Human Preferences: Mitigating Token Count Bias in Language Model Assessments
by: Daynauth, Roland, et al.
Published: (2024)

Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments
by: Zhou, Han, et al.
Published: (2024)

SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
by: Yu, Donglei, et al.
Published: (2025)

How does Misinformation Affect Large Language Model Behaviors and Preferences?
by: Peng, Miao, et al.
Published: (2025)

Multiplayer Nash Preference Optimization
by: Wu, Fang, et al.
Published: (2025)