Cover Image

TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shing, Makoto, Misaki, Kou, Bao, Han, Yokoi, Sho, Akiba, Takuya
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

UnMaskFork: Test-Time Scaling for Masked Diffusion via Deterministic Action Branching
by: Misaki, Kou, et al.
Published: (2026)

DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation
by: Shing, Makoto, et al.
Published: (2025)

String Seed of Thought: Prompting LLMs for Distribution-Faithful and Diverse Generation
by: Misaki, Kou, et al.
Published: (2025)

LLM-NEO: Parameter Efficient Knowledge Distillation for Large Language Models
by: Yang, Runming, et al.
Published: (2024)

KDFlow: A User-Friendly and Efficient Knowledge Distillation Framework for Large Language Models
by: Zhang, Songming, et al.
Published: (2026)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation
by: Zhang, Songming, et al.
Published: (2025)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
by: Tian, Yijun, et al.
Published: (2024)

Distilling Multi-Scale Knowledge for Event Temporal Relation Extraction
by: Yao, Hao-Ren, et al.
Published: (2022)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
by: Yu, Fengming, et al.
Published: (2025)

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
by: Zhong, Ming, et al.
Published: (2023)

Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective
by: Kajitsuka, Tokio, et al.
Published: (2026)

Leveraging Zero-Shot Prompting for Efficient Language Model Distillation
by: Vöge, Lukas, et al.
Published: (2024)

A Dual-Space Framework for General Knowledge Distillation of Large Language Models
by: Zhang, Xue, et al.
Published: (2025)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
by: Chen, Jennifer, et al.
Published: (2025)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
by: Hu, Yuezhou, et al.
Published: (2025)

SDMPrune: Self-Distillation MLP Pruning for Efficient Large Language Models
by: Zhu, Hourun, et al.
Published: (2025)

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models
by: Rajaee, Sara, et al.
Published: (2024)

Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Models
by: Kim, Gyeongman, et al.
Published: (2025)

GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models
by: Liao, Ruotong, et al.
Published: (2023)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
by: Kim, Gyeongman, et al.
Published: (2024)

Parameter Efficient Diverse Paraphrase Generation Using Sequence-Level Knowledge Distillation
by: Jayawardena, Lasal, et al.
Published: (2024)

Learning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer
by: Dehigahawattage, Nilushika Udayangani Hewa, et al.
Published: (2026)

KGLens: Towards Efficient and Effective Knowledge Probing of Large Language Models with Knowledge Graphs
by: Zheng, Shangshang, et al.
Published: (2023)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration
by: Wang, Fali, et al.
Published: (2024)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models
by: Bhardwaj, Kartikeya, et al.
Published: (2024)

Being Strong Progressively! Enhancing Knowledge Distillation of Large Language Models through a Curriculum Learning Framework
by: Liu, Lingyuan, et al.
Published: (2025)

zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
by: Ding, Zifeng, et al.
Published: (2023)

Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
by: Nakamura, Taishi, et al.
Published: (2025)

LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation
by: Yamada, Ikuya, et al.
Published: (2024)

LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models
by: Veldanda, Akshaj Kumar, et al.
Published: (2024)

ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
by: Lian, Long, et al.
Published: (2025)

Structured Agent Distillation for Large Language Model
by: Liu, Jun, et al.
Published: (2025)

Large Language Models Explore by Latent Distilling
by: Zeng, Yuanhao, et al.
Published: (2026)

Evolutionary Contrastive Distillation for Language Model Alignment
by: Katz-Samuels, Julian, et al.
Published: (2024)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Zipfian Whitening
by: Yokoi, Sho, et al.
Published: (2024)