Cover Image

Enhancing Knowledge Distillation for LLMs with Response-Priming Prompting

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Goyal, Vijay, Khan, Mustafa, Tirupati, Aprameya, Saini, Harveer, Lam, Michael, Zhu, Kevin
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interaction
by: Sandwar, Vivaan, et al.
Published: (2025)

Updating Parametric Knowledge with Context Distillation Retains Post-Training Capabilities
by: Padmanabhan, Shankar, et al.
Published: (2026)

Enhancing Depression Diagnosis with Chain-of-Thought Prompting
by: Shi, Elysia, et al.
Published: (2024)

Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study
by: Roy, Aniruddha, et al.
Published: (2024)

KDH-MLTC: Knowledge Distillation for Healthcare Multi-Label Text Classification
by: Sakai, Hajar, et al.
Published: (2025)

Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
by: Jin, Ruihan, et al.
Published: (2026)

Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching
by: Li, Mingzhe, et al.
Published: (2025)

DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models
by: Kim, Sungnyun, et al.
Published: (2024)

Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs
by: Boizard, Nicolas, et al.
Published: (2024)

KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs
by: Liang, Buyun, et al.
Published: (2025)

Memorization vs. Reasoning: Updating LLMs with New Knowledge
by: Li, Aochong Oliver, et al.
Published: (2025)

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
by: Zhong, Qihuang, et al.
Published: (2022)

UNDO: Understanding Distillation as Optimization
by: Jain, Kushal, et al.
Published: (2025)

Trace-of-Thought Prompting: Investigating Prompt-Based Knowledge Distillation Through Question Decomposition
by: McDonald, Tyler, et al.
Published: (2025)

Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models
by: Miao, Ziqi, et al.
Published: (2025)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
by: Chen, Jennifer, et al.
Published: (2025)

Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation
by: Csizmadia, Daniel, et al.
Published: (2025)

Efficient Knowledge Injection in LLMs via Self-Distillation
by: Kujanpää, Kalle, et al.
Published: (2024)

Towards Reliable Latent Knowledge Estimation in LLMs: Zero-Prompt Many-Shot Based Factual Knowledge Extraction
by: Wu, Qinyuan, et al.
Published: (2024)

Ratatouille: A tool for Novel Recipe Generation
by: Goel, Mansi, et al.
Published: (2022)

Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning
by: Chen, Qizhou, et al.
Published: (2024)

Contrastive Distillation of Emotion Knowledge from LLMs for Zero-Shot Emotion Recognition
by: Niu, Minxue, et al.
Published: (2025)

Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs
by: Gao, Xin, et al.
Published: (2025)

Hybrid Policy Distillation for LLMs
by: Zhu, Wenhong, et al.
Published: (2026)

Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping
by: Chen, Yijie, et al.
Published: (2025)

SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
by: Huang, Haiduo, et al.
Published: (2025)

Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
by: Hamman, Faisal, et al.
Published: (2025)

Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety
by: Zhang, Max, et al.
Published: (2025)

Distill-C: Enhanced NL2SQL via Distilled Customization with LLMs
by: Hoang, Cong Duy Vu, et al.
Published: (2025)

MASH: Modeling Abstention via Selective Help-Seeking
by: Gul, Mustafa Omer, et al.
Published: (2025)

Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge
by: Fu, Jinlan, et al.
Published: (2024)

MAGIC-Enhanced Keyword Prompting for Zero-Shot Audio Captioning with CLIP Models
by: Govindarajan, Vijay, et al.
Published: (2025)

CourseGPT-zh: an Educational Large Language Model Based on Knowledge Distillation Incorporating Prompt Optimization
by: Qu, Zheyan, et al.
Published: (2024)

Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
by: Rao, Jun, et al.
Published: (2024)

SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics
by: You, Zhiwen, et al.
Published: (2024)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
by: Kim, Gyeongman, et al.
Published: (2024)

Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMs
by: Li, Xiang, et al.
Published: (2025)

IOLBENCH: Benchmarking LLMs on Linguistic Reasoning
by: Goyal, Satyam, et al.
Published: (2025)

Accuracy and Consistency of LLMs in the Registered Dietitian Exam: The Impact of Prompt Engineering and Knowledge Retrieval
by: Azimi, Iman, et al.
Published: (2024)

IAO Prompting: Making Knowledge Flow Explicit in LLMs through Structured Reasoning Templates
by: Diallo, Aissatou, et al.
Published: (2025)