Imagen de Portada

Task-Specific Knowledge Distillation via Intermediate Probes

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Brown, Ryan, Russell, Chris
Formato:	Preprint
Publicado:	2026
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Outlier Dimensions Encode Task-Specific Knowledge
por: Rudman, William, et al.
Publicado: (2023)

TreeMatch: A Fully Unsupervised WSD System Using Dependency Knowledge on a Specific Domain
por: Tran, Andrew, et al.
Publicado: (2025)

DSG-KD: Knowledge Distillation from Domain-Specific to General Language Models
por: Cho, Sangyeon, et al.
Publicado: (2024)

Beyond Output Critique: Self-Correction via Task Distillation
por: Rahmani, Hossein A., et al.
Publicado: (2026)

PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation
por: Fan, Tao, et al.
Publicado: (2025)

Domain Specific Specialization in Low-Resource Settings: The Efficacy of Offline Response-Based Knowledge Distillation in Large Language Models
por: Aslan, Erdem, et al.
Publicado: (2026)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
por: Zhou, Yongchao, et al.
Publicado: (2023)

Intermediate-Task Transfer Learning: Leveraging Sarcasm Detection for Stance Detection
por: Nkhata, Gibson, et al.
Publicado: (2025)

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
por: Tan, Zhen, et al.
Publicado: (2026)

Decoder-based Sense Knowledge Distillation
por: Wang, Qitong, et al.
Publicado: (2026)

AdaSwitch: Balancing Exploration and Guidance in Knowledge Distillation via Adaptive Switching
por: Peng, Jingyu, et al.
Publicado: (2025)

Self-Knowledge Distillation for Learning Ambiguity
por: Park, Hancheol, et al.
Publicado: (2024)

SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning
por: Zhao, Chenyang, et al.
Publicado: (2024)

LLM-Oriented Token-Adaptive Knowledge Distillation
por: Xie, Xurong, et al.
Publicado: (2025)

EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
por: Wang, Chengyu, et al.
Publicado: (2025)

Probing LLM Hallucination from Within: Perturbation-Driven Approach via Internal Knowledge
por: Lee, Seongmin, et al.
Publicado: (2024)

Compact Language Models via Pruning and Knowledge Distillation
por: Muralidharan, Saurav, et al.
Publicado: (2024)

Provable Benefits of Task-Specific Prompts for In-context Learning
por: Chang, Xiangyu, et al.
Publicado: (2025)

AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning
por: Cui, Jiaxi, et al.
Publicado: (2024)

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation
por: Zhou, Yuhang, et al.
Publicado: (2024)

Dual-Space Knowledge Distillation for Large Language Models
por: Zhang, Songming, et al.
Publicado: (2024)

Multimodal Commonsense Knowledge Distillation for Visual Question Answering
por: Yang, Shuo, et al.
Publicado: (2024)

LKD-KGC: Domain-Specific KG Construction via LLM-driven Knowledge Dependency Parsing
por: Sun, Jiaqi, et al.
Publicado: (2025)

URL: Universal Referential Knowledge Linking via Task-instructed Representation Compression
por: Li, Zhuoqun, et al.
Publicado: (2024)

LLMs to Support a Domain Specific Knowledge Assistant
por: Lovin, Maria-Flavia
Publicado: (2025)

KBAlign: Efficient Self Adaptation on Specific Knowledge Bases
por: Zeng, Zheni, et al.
Publicado: (2024)

Can LLMs Generate High-Quality Task-Specific Conversations?
por: Li, Shengqi, et al.
Publicado: (2025)

Pedagogically-Inspired Data Synthesis for Language Model Knowledge Distillation
por: He, Bowei, et al.
Publicado: (2026)

Cross-Modal Knowledge Distillation for Speech Large Language Models
por: Wang, Enzhi, et al.
Publicado: (2025)

Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
por: Latif, Ehsan, et al.
Publicado: (2023)

Distilling Event Sequence Knowledge From Large Language Models
por: Wadhwa, Somin, et al.
Publicado: (2024)

SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
por: Huang, Haiduo, et al.
Publicado: (2025)

Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
por: Zhang, Songming, et al.
Publicado: (2023)

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs
por: Chen, Yurun, et al.
Publicado: (2025)

Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
por: Chen, Wei-Rui, et al.
Publicado: (2025)

AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement
por: Singh, Shivam, et al.
Publicado: (2025)

Probing the Capacity of Language Model Agents to Operationalize Disparate Experiential Context Despite Distraction
por: George, Sonny, et al.
Publicado: (2024)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
por: Yu, Fengming, et al.
Publicado: (2025)

DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation
por: Jiang, Bo
Publicado: (2026)

Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention
por: Lin, Wenye, et al.
Publicado: (2026)