Copertina

Direct Preference Knowledge Distillation for Large Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Yixing, Gu, Yuxian, Dong, Li, Wang, Dequan, Cheng, Yu, Wei, Furu
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

MiniLLM: On-Policy Distillation of Large Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2023)

Towards Optimal Learning of Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2024)

Data Selection via Optimal Control for Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2024)

Self-Boosting Large Language Models with Synthetic Preference Data
di: Dong, Qingxiu, et al.
Pubblicazione: (2024)

Instruction Pre-Training: Language Models are Supervised Multitask Learners
di: Cheng, Daixuan, et al.
Pubblicazione: (2024)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2024)

On-Policy Context Distillation for Language Models
di: Ye, Tianzhu, et al.
Pubblicazione: (2026)

Black-Box On-Policy Distillation of Large Language Models
di: Ye, Tianzhu, et al.
Pubblicazione: (2025)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
di: Xing, Wang, et al.
Pubblicazione: (2026)

Adapting Large Language Models to Domains via Reading Comprehension
di: Cheng, Daixuan, et al.
Pubblicazione: (2023)

A Survey on Knowledge Distillation of Large Language Models
di: Xu, Xiaohan, et al.
Pubblicazione: (2024)

Capturing Nuanced Preferences: Preference-Aligned Distillation for Small Language Models
di: Gu, Yanggan, et al.
Pubblicazione: (2025)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
di: Fang, Luyang, et al.
Pubblicazione: (2025)

Learning to Retrieve In-Context Examples for Large Language Models
di: Wang, Liang, et al.
Pubblicazione: (2023)

Establishing Knowledge Preference in Language Models
di: Zhou, Sizhe, et al.
Pubblicazione: (2024)

Knowledge Distillation of Black-Box Large Language Models
di: Chen, Hongzhan, et al.
Pubblicazione: (2024)

MT-PATCHER: Selective and Extendable Knowledge Distillation from Large Language Models for Machine Translation
di: Li, Jiahuan, et al.
Pubblicazione: (2024)

Cross-Modal Knowledge Distillation for Speech Large Language Models
di: Wang, Enzhi, et al.
Pubblicazione: (2025)

Online Experiential Learning for Language Models
di: Ye, Tianzhu, et al.
Pubblicazione: (2026)

Kosmos-G: Generating Images in Context with Multimodal Large Language Models
di: Pan, Xichen, et al.
Pubblicazione: (2023)

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
di: Li, Haoran, et al.
Pubblicazione: (2024)

BitNet Distillation
di: Wu, Xun, et al.
Pubblicazione: (2025)

D2LLM: Decomposed and Distilled Large Language Models for Semantic Search
di: Liao, Zihan, et al.
Pubblicazione: (2024)

Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
di: Wu, Wenshan, et al.
Pubblicazione: (2024)

Computer Environments Elicit General Agentic Intelligence in LLMs
di: Cheng, Daixuan, et al.
Pubblicazione: (2026)

PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
di: Zhang, Rongzhi, et al.
Pubblicazione: (2024)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
di: Wang, Hongyu, et al.
Pubblicazione: (2024)

DDK: Distilling Domain Knowledge for Efficient Large Language Models
di: Liu, Jiaheng, et al.
Pubblicazione: (2024)

LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
di: Li, Dongheng, et al.
Pubblicazione: (2024)

DARE: Diffusion Large Language Models Alignment and Reinforcement Executor
di: Yang, Jingyi, et al.
Pubblicazione: (2026)

Distilling Rule-based Knowledge into Large Language Models
di: Yang, Wenkai, et al.
Pubblicazione: (2023)

Formality is Favored: Unraveling the Learning Preferences of Large Language Models on Data with Conflicting Knowledge
di: Li, Jiahuan, et al.
Pubblicazione: (2024)

MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
di: Jiang, Mohan, et al.
Pubblicazione: (2025)

Contextualization Distillation from Large Language Model for Knowledge Graph Completion
di: Li, Dawei, et al.
Pubblicazione: (2024)

Revisiting Knowledge Distillation for Autoregressive Language Models
di: Zhong, Qihuang, et al.
Pubblicazione: (2024)

Multi-Granularity Semantic Revision for Large Language Model Distillation
di: Liu, Xiaoyu, et al.
Pubblicazione: (2024)

Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models
di: Wang, Shuo, et al.
Pubblicazione: (2024)

SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models
di: Koo, Jahyun, et al.
Pubblicazione: (2024)

Evolving Knowledge Distillation with Large Language Models and Active Learning
di: Liu, Chengyuan, et al.
Pubblicazione: (2024)

MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery
di: Lu, Feihong, et al.
Pubblicazione: (2024)