Copertina

m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lo, Ka Man, Liang, Yiming, Du, Wenyu, Fan, Yuantao, Wang, Zili, Huang, Wenhao, Ma, Lei, Fu, Jie
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Computer Vision and Pattern Recognition
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling
di: Wang, Yu, et al.
Pubblicazione: (2022)

EA-KD: Entropy-based Adaptive Knowledge Distillation
di: Su, Chi-Ping, et al.
Pubblicazione: (2023)

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation
di: Liu, Ruiping, et al.
Pubblicazione: (2022)

TopKD: Top-scaled Knowledge Distillation
di: Wang, Qi, et al.
Pubblicazione: (2025)

FreeKD: Knowledge Distillation via Semantic Frequency Prompt
di: Zhang, Yuan, et al.
Pubblicazione: (2023)

MoKD: Multi-Task Optimization for Knowledge Distillation
di: Hayder, Zeeshan, et al.
Pubblicazione: (2025)

BD-KD: Balancing the Divergences for Online Knowledge Distillation
di: Amara, Ibtihel, et al.
Pubblicazione: (2022)

M$^2$CD: A Unified MultiModal Framework for Optical-SAR Change Detection with Mixture of Experts and Self-Distillation
di: Liu, Ziyuan, et al.
Pubblicazione: (2025)

REACT-KD: Region-Aware Cross-modal Topological Knowledge Distillation for Interpretable Medical Image Classification
di: Chen, Hongzhao, et al.
Pubblicazione: (2025)

DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer
di: Huang, Haiduo, et al.
Pubblicazione: (2025)

MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders
di: Cao, Jiajun, et al.
Pubblicazione: (2025)

A Closer Look into Mixture-of-Experts in Large Language Models
di: Lo, Ka Man, et al.
Pubblicazione: (2024)

ACAM-KD: Adaptive and Cooperative Attention Masking for Knowledge Distillation
di: Lan, Qizhen, et al.
Pubblicazione: (2025)

CrossKD: Cross-Head Knowledge Distillation for Object Detection
di: Wang, Jiabao, et al.
Pubblicazione: (2023)

Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models
di: Sun, Haoyi, et al.
Pubblicazione: (2026)

Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling
di: Mekonnen, Kidist Amde, et al.
Pubblicazione: (2024)

AI-KD: Adversarial learning and Implicit regularization for self-Knowledge Distillation
di: Kim, Hyungmin, et al.
Pubblicazione: (2022)

VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition
di: Zhang, Zaiwei, et al.
Pubblicazione: (2024)

MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition
di: Caldeira, Eduarda, et al.
Pubblicazione: (2024)

TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation
di: Choi, Sangwon, et al.
Pubblicazione: (2024)

StableKD: Breaking Inter-block Optimization Entanglement for Stable Knowledge Distillation
di: Kao, Shiu-hong, et al.
Pubblicazione: (2023)

PromptKD: Unsupervised Prompt Distillation for Vision-Language Models
di: Li, Zheng, et al.
Pubblicazione: (2024)

CLIP-KD: An Empirical Study of CLIP Model Distillation
di: Yang, Chuanguang, et al.
Pubblicazione: (2023)

AgriKD: Cross-Architecture Knowledge Distillation for Efficient Leaf Disease Classification
di: Le, Minh-Dung, et al.
Pubblicazione: (2026)

ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images
di: Ge, Hongyu, et al.
Pubblicazione: (2025)

LIB-KD: Teaching Inductive Bias for Efficient Vision Transformer Distillation and Compression
di: Habib, Gousia, et al.
Pubblicazione: (2023)

KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
di: Nourbakhsh, Erfan, et al.
Pubblicazione: (2025)

DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models
di: Kim, Sungnyun, et al.
Pubblicazione: (2024)

Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model
di: Feng, Qianhan, et al.
Pubblicazione: (2024)

AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation
di: Tang, Zihao, et al.
Pubblicazione: (2024)

AI-KD: Towards Alignment Invariant Face Image Quality Assessment Using Knowledge Distillation
di: Babnik, Žiga, et al.
Pubblicazione: (2024)

CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation
di: Lee, Jungsoo, et al.
Pubblicazione: (2025)

ComKD-CLIP: Comprehensive Knowledge Distillation for Contrastive Language-Image Pre-traning Model
di: Chen, Yifan, et al.
Pubblicazione: (2024)

DiffKD-DCIS: Predicting Upgrade of Ductal Carcinoma In Situ with Diffusion Augmentation and Knowledge Distillation
di: Li, Tao, et al.
Pubblicazione: (2026)

CanKD: Cross-Attention-based Non-local operation for Feature-based Knowledge Distillation
di: Sun, Shizhe, et al.
Pubblicazione: (2025)

CMT: Cross Modulation Transformer with Hybrid Loss for Pansharpening
di: Shu, Wen-Jie, et al.
Pubblicazione: (2024)

Analyzing Transformer Models and Knowledge Distillation Approaches for Image Captioning on Edge AI
di: Kwok, Wing Man Casca, et al.
Pubblicazione: (2025)

Efficient Temporal Sentence Grounding in Videos with Multi-Teacher Knowledge Distillation
di: Liang, Renjie, et al.
Pubblicazione: (2023)

MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction
di: Yan, Ziyang, et al.
Pubblicazione: (2025)

GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation
di: Banerjee, Ayan, et al.
Pubblicazione: (2024)