Cover Image

TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Guan, Yushi, Ohene-Agyei, Jeanine, Kwan, Daniel, Dandurand, Jean Sebastien, Zhang, Yifei, Vijaykumar, Nandita
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

INRet: A General Framework for Accurate Retrieval of INRs for Shapes
by: Guan, Yushi, et al.
Published: (2025)

ShiftKD: Benchmarking Knowledge Distillation under Distribution Shift
by: Zhang, Songming, et al.
Published: (2023)

Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural Compressor
by: Dagli, Rishit, et al.
Published: (2025)

KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
by: Azimi, Rambod, et al.
Published: (2024)

BicKD: Bilateral Contrastive Knowledge Distillation
by: Zhu, Jiangnan, et al.
Published: (2026)

PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
by: Kim, Gyeongman, et al.
Published: (2024)

Relational Knowledge Distillation Using Fine-tuned Function Vectors
by: Kang, Andrea, et al.
Published: (2026)

FlyKD: Graph Knowledge Distillation on the Fly with Curriculum Learning
by: Ku, Eugene
Published: (2024)

RMT-KD: Random Matrix Theoretic Causal Knowledge Distillation
by: Ettori, Davide, et al.
Published: (2025)

DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling
by: Gao, Yubo, et al.
Published: (2025)

S^2-KD: Semantic-Spectral Knowledge Distillation Spatiotemporal Forecasting
by: Wang, Wenshuo, et al.
Published: (2025)

Sparse Code Uplifting for Efficient 3D Language Gaussian Splatting
by: Budimir, Lovre Antonio, et al.
Published: (2026)

DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer
by: Huang, Haiduo, et al.
Published: (2025)

SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation
by: Xu, Haotian, et al.
Published: (2025)

HPM-KD: Hierarchical Progressive Multi-Teacher Framework for Knowledge Distillation and Efficient Model Compression
by: Haase, Gustavo Coelho, et al.
Published: (2025)

Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling
by: Mekonnen, Kidist Amde, et al.
Published: (2024)

Communication-Aware Knowledge Distillation for Federated LLM Fine-Tuning over Wireless Networks
by: Zhang, Xinlu, et al.
Published: (2025)

AI-KD: Adversarial learning and Implicit regularization for self-Knowledge Distillation
by: Kim, Hyungmin, et al.
Published: (2022)

ACS: Concurrent Kernel Execution on Irregular, Input-Dependent Computational Graphs
by: Durvasula, Sankeerth, et al.
Published: (2024)

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
by: Nair, Lakshmi
Published: (2024)

DEQuify your force field: More efficient simulations using deep equilibrium models
by: Burger, Andreas, et al.
Published: (2025)

HyperKD: Distilling Cross-Spectral Knowledge in Masked Autoencoders via Inverse Domain Shift with Spatial-Aware Masking and Specialized Loss
by: Matin, Abdul, et al.
Published: (2025)

AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation
by: Tang, Zihao, et al.
Published: (2024)

BackWeak: Backdooring Knowledge Distillation Simply with Weak Triggers and Fine-tuning
by: Wang, Shanmin, et al.
Published: (2025)

Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?
by: Hościłowicz, Jakub, et al.
Published: (2023)

MemKD: Memory-Discrepancy Knowledge Distillation for Efficient Time Series Classification
by: Udayangani, Nilushika, et al.
Published: (2026)

m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers
by: Lo, Ka Man, et al.
Published: (2024)

Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer
by: Du, Guodong, et al.
Published: (2025)

Efficient Model Development through Fine-tuning Transfer
by: Lin, Pin-Jie, et al.
Published: (2025)

C2G-KD: PCA-Constrained Generator for Data-Free Knowledge Distillation
by: Bengtsson, Magnus, et al.
Published: (2025)

TabKD: Tabular Knowledge Distillation through Interaction Diversity of Learned Feature Bins
by: Pereira, Shovon Niverd, et al.
Published: (2026)

AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting
by: Ganguly, Shreyan, et al.
Published: (2024)

FedeKD: Energy-Based Gating for Robust Federated Knowledge Distillation under Heterogeneous Settings
by: Nguyen, Quang-Huy, et al.
Published: (2026)

Proteus: Preserving Model Confidentiality during Graph Optimizations
by: Gao, Yubo, et al.
Published: (2024)

Federated Knowledge Transfer Fine-tuning Large Server Model with Resource-Constrained IoT Clients
by: Chen, Shaoyuan, et al.
Published: (2024)

KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
by: Nourbakhsh, Erfan, et al.
Published: (2025)

MERG3R: A Divide-and-Conquer Approach to Large-Scale Neural Visual Geometry
by: Cheng, Leo Kaixuan, et al.
Published: (2026)

The Magic Correlations: Understanding Knowledge Transfer from Pretraining to Supervised Fine-Tuning
by: Fan, Simin, et al.
Published: (2026)

TuneComp: Joint Fine-tuning and Compression for Large Foundation Models
by: Chen, Xiangyu, et al.
Published: (2025)

CRAFT: Fine-Grained Cost-Aware Expert Replication For Efficient Mixture-of-Experts Serving
by: Zhao, Adrian, et al.
Published: (2026)