Image de couverture de livre

Revisiting Knowledge Distillation for Autoregressive Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhong, Qihuang, Ding, Liang, Shen, Li, Liu, Juhua, Du, Bo, Tao, Dacheng
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL
par: Zhong, Qihuang, et autres
Publié: (2024)

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
par: Zhong, Qihuang, et autres
Publié: (2022)

ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding
par: Zhong, Qihuang, et autres
Publié: (2024)

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation
par: Zhong, Qihuang, et autres
Publié: (2022)

Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning
par: Zhong, Qihuang, et autres
Publié: (2025)

KaFT: Knowledge-aware Fine-tuning for Boosting LLMs' Domain-specific Question-Answering Performance
par: Zhong, Qihuang, et autres
Publié: (2025)

Better, Faster: Harnessing Self-Improvement in Large Reasoning Models
par: Zhong, Qihuang, et autres
Publié: (2026)

Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs
par: Chen, Kunfeng, et autres
Publié: (2026)

Iterative Data Generation with Large Language Models for Aspect-based Sentiment Analysis
par: Zhong, Qihuang, et autres
Publié: (2024)

Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
par: Rao, Jun, et autres
Publié: (2024)

Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems
par: Zhong, Qihuang, et autres
Publié: (2024)

ConRAG: Consensus-Driven Multi-View Retrieval for Multi-Hop Question Answering
par: Zhu, Yikai, et autres
Publié: (2026)

Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
par: Zhong, Qihuang, et autres
Publié: (2026)

LogicOCR: Do Your Large Multimodal Models Excel at Logical Reasoning on Text-Rich Images?
par: Ye, Maoyuan, et autres
Publié: (2025)

Revisiting Catastrophic Forgetting in Large Language Model Tuning
par: Li, Hongyu, et autres
Publié: (2024)

A Survey on Knowledge Distillation of Large Language Models
par: Xu, Xiaohan, et autres
Publié: (2024)

OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
par: Wang, Shuai, et autres
Publié: (2024)

Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing
par: Wu, Yuchen, et autres
Publié: (2025)

Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
par: Wu, Yuchen, et autres
Publié: (2025)

Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
par: Xiao, Yisong, et autres
Publié: (2025)

Edit Once, Update Everywhere: A Simple Framework for Cross-Lingual Knowledge Synchronization in LLMs
par: Wu, Yuchen, et autres
Publié: (2025)

Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning
par: Xia, Tianle, et autres
Publié: (2024)

WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
par: Wang, Wenbin, et autres
Publié: (2024)

Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
par: Peng, Keqin, et autres
Publié: (2025)

Revisiting Demonstration Selection Strategies in In-Context Learning
par: Peng, Keqin, et autres
Publié: (2024)

Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer
par: Liu, Boan, et autres
Publié: (2023)

Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning
par: Zan, Changtong, et autres
Publié: (2024)

The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check
par: Lu, Qingyu, et autres
Publié: (2026)

Entropy-Guided Watermarking for LLMs: A Test-Time Framework for Robust and Traceable Text Generation
par: Cai, Shizhan, et autres
Publié: (2025)

Distilling Implicit Multimodal Knowledge into Large Language Models for Zero-Resource Dialogue Generation
par: Zhang, Bo, et autres
Publié: (2024)

Uncertainty Aware Learning for Language Model Alignment
par: Wang, Yikun, et autres
Publié: (2024)

Retrieval-Augmented Perception: High-Resolution Image Perception Meets Visual RAG
par: Wang, Wenbin, et autres
Publié: (2025)

Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models
par: Lu, Qingyu, et autres
Publié: (2023)

Model Hemorrhage and the Robustness Limits of Large Language Models
par: Ma, Ziyang, et autres
Publié: (2025)

Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation
par: Huang, Chenyang, et autres
Publié: (2025)

Tool Retrieval Bridge: Aligning Vague Instructions with Retriever Preferences via Bridge Model
par: Chen, Kunfeng, et autres
Publié: (2026)

DDK: Distilling Domain Knowledge for Efficient Large Language Models
par: Liu, Jiaheng, et autres
Publié: (2024)

LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classification
par: Song, Yiping, et autres
Publié: (2024)

Intention Analysis Makes LLMs A Good Jailbreak Defender
par: Zhang, Yuqi, et autres
Publié: (2024)

Stable Knowledge Editing in Large Language Models
par: Wei, Zihao, et autres
Publié: (2024)