Cover Image

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vöge, Lukas, Gurgul, Vincent, Lessmann, Stefan
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Prompting Large Language Models for Zero-Shot Clinical Prediction with Structured Longitudinal Electronic Health Record Data
by: Zhu, Yinghao, et al.
Published: (2024)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
by: Kim, Gyeongman, et al.
Published: (2024)

Agent Instructs Large Language Models to be General Zero-Shot Reasoners
by: Crispino, Nicholas, et al.
Published: (2023)

ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
by: Lin, Yujie, et al.
Published: (2026)

BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
by: Zhu, Alan, et al.
Published: (2025)

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic
by: Zhao, Xufeng, et al.
Published: (2023)

The State of Generative AI in Software Development: Insights from Literature and a Developer Survey
by: Gurgul, Vincent, et al.
Published: (2026)

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models
by: Bandarkar, Lucas, et al.
Published: (2024)

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
by: Toukmaji, Christopher
Published: (2024)

Automatic Prompt Optimization with Prompt Distillation
by: Dyagin, Ernest A., et al.
Published: (2025)

Token-Efficient Leverage Learning in Large Language Models
by: Zeng, Yuanhao, et al.
Published: (2024)

LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction
by: Nagar, Aishik, et al.
Published: (2024)

Variational Quantum Circuit-Based Reinforcement Learning for Dynamic Portfolio Optimization
by: Gurgul, Vincent, et al.
Published: (2026)

Crossing New Frontiers: Knowledge-Augmented Large Language Model Prompting for Zero-Shot Text-Based De Novo Molecule Design
by: Srinivas, Sakhinana Sagar, et al.
Published: (2024)

zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
by: Ding, Zifeng, et al.
Published: (2023)

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare
by: Gondara, Lovedeep, et al.
Published: (2025)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
by: Zhu, Hourun, et al.
Published: (2025)

Language Models as Zero-Shot Trajectory Generators
by: Kwon, Teyun, et al.
Published: (2023)

Leveraging Large Language Models with Chain-of-Thought and Prompt Engineering for Traffic Crash Severity Analysis and Inference
by: Zhen, Hao, et al.
Published: (2024)

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes
by: Gallegos, Isabel O., et al.
Published: (2024)

GLiREL -- Generalist Model for Zero-Shot Relation Extraction
by: Boylan, Jack, et al.
Published: (2025)

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
by: Shing, Makoto, et al.
Published: (2025)

TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification
by: Dissanayake, Pasan, et al.
Published: (2025)

Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models
by: Ozyurt, Yilmazcan, et al.
Published: (2023)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
by: Kumar, Sachin
Published: (2026)

Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
by: Ali, Muhammad Asif, et al.
Published: (2024)

Meta-Adaptive Prompt Distillation for Few-Shot Visual Question Answering
by: Gupta, Akash, et al.
Published: (2025)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
by: Zhang, Songming, et al.
Published: (2026)

Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification
by: Hu, Yibo, et al.
Published: (2023)

Understanding and Mitigating the Uncertainty in Zero-Shot Translation
by: Wang, Wenxuan, et al.
Published: (2022)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

Few-Shot Recalibration of Language Models
by: Li, Xiang Lisa, et al.
Published: (2024)

Language Representation Favored Zero-Shot Cross-Domain Cognitive Diagnosis
by: Liu, Shuo, et al.
Published: (2025)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Evolutionary Contrastive Distillation for Language Model Alignment
by: Katz-Samuels, Julian, et al.
Published: (2024)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition
by: Yan, Yuxuan, et al.
Published: (2024)