Cover Image

LIB-KD: Teaching Inductive Bias for Efficient Vision Transformer Distillation and Compression

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Habib, Gousia, Saleem, Tausifa Jan, Malik, Ishfaq Ahmad, Lall, Brejesh
Format:	Preprint
Published:	2023
Subjects:	Computer Vision and Pattern Recognition Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Knowledge Distillation in Vision Transformers: A Critical Review
by: Habib, Gousia, et al.
Published: (2023)

Optimizing Vision Transformers with Data-Free Knowledge Transfer
by: Habib, Gousia, et al.
Published: (2024)

A Comprehensive Review of Knowledge Distillation in Computer Vision
by: Habib, Gousia, et al.
Published: (2024)

CFL-SparseMed: Communication-Efficient Federated Learning for Medical Imaging with Top-k Sparse Updates
by: Habib, Gousia, et al.
Published: (2025)

Exploring the Efficacy of Group-Normalization in Deep Learning Models for Alzheimer's Disease Classification
by: Habib, Gousia, et al.
Published: (2024)

Harnessing The Power of Attention For Patch-Based Biomedical Image Classification
by: Habib, Gousia, et al.
Published: (2024)

Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruning
by: Saleem, Tausifa Jan, et al.
Published: (2024)

Leveraging band diversity for feature selection in EO data
by: Hussain, Sadia, et al.
Published: (2025)

Single Stage Warped Cloth Learning and Semantic-Contextual Attention Feature Fusion for Virtual TryOn
by: Pathak, Sanhita, et al.
Published: (2023)

GraVITON: Graph based garment warping with attention guided inversion for Virtual-tryon
by: Pathak, Sanhita, et al.
Published: (2024)

DiffSTR: Controlled Diffusion Models for Scene Text Removal
by: Pathak, Sanhita, et al.
Published: (2024)

Deep Learning-Based Automated Segmentation of Uterine Myomas
by: Saleem, Tausifa Jan, et al.
Published: (2025)

SpectralKD: A Unified Framework for Interpreting and Distilling Vision Transformers via Spectral Analysis
by: Tian, Huiyuan, et al.
Published: (2024)

Optimizing Vision Transformers With Data‐Free Knowledge Transfer
by: Gousia Habib, et al.
Published: (2026)

Stride-Net: Fairness-Aware Disentangled Representation Learning for Chest X-Ray Diagnosis
by: Rashid, Darakshan, et al.
Published: (2026)

m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers
by: Lo, Ka Man, et al.
Published: (2024)

HamVision: Hamiltonian Dynamics as Inductive Bias for Medical Image Analysis
by: Mabrok, Mohamed A
Published: (2026)

Dual Thinking and Logical Processing -- Are Multi-modal Large Language Models Closing the Gap with Human Vision ?
by: Dayanandan, Kailas, et al.
Published: (2024)

SAFE-KD: Risk-Controlled Early-Exit Distillation for Vision Backbones
by: Khazem, Salim
Published: (2026)

Unified Multi-Dataset Training for TBPS
by: Chatterjee, Nilanjana, et al.
Published: (2026)

The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
by: Son, Seungwoo, et al.
Published: (2023)

DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosis
by: Saeed, Numan, et al.
Published: (2025)

ScaleKD: Strong Vision Transformers Could Be Excellent Teachers
by: Fan, Jiawei, et al.
Published: (2024)

Continual Segmentation under Joint Nonstationarity
by: Pandey, Prashant, et al.
Published: (2026)

KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
by: Nourbakhsh, Erfan, et al.
Published: (2025)

EM-KD: Distilling Efficient Multimodal Large Language Model with Unbalanced Vision Tokens
by: Feng, Ze, et al.
Published: (2025)

FerKD: Surgical Label Adaptation for Efficient Distillation
by: Shen, Zhiqiang
Published: (2023)

AI-KD: Adversarial learning and Implicit regularization for self-Knowledge Distillation
by: Kim, Hyungmin, et al.
Published: (2022)

PromptKD: Unsupervised Prompt Distillation for Vision-Language Models
by: Li, Zheng, et al.
Published: (2024)

KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling
by: Wang, Yu, et al.
Published: (2022)

MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answering
by: Shaaban, Mai A., et al.
Published: (2025)

MedSPOT: A Workflow-Aware Sequential Grounding Benchmark for Clinical GUI
by: Shakeel, Rozain, et al.
Published: (2026)

Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations
by: Shi, Kexuan, et al.
Published: (2024)

TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation
by: Liu, Ruiping, et al.
Published: (2022)

AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation
by: Tang, Zihao, et al.
Published: (2024)

Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models
by: Sun, Haoyi, et al.
Published: (2026)

GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation
by: Banerjee, Ayan, et al.
Published: (2024)

Garbage Vulnerable Point Monitoring using IoT and Computer Vision
by: Kumar, R., et al.
Published: (2025)

A Comprehensive Survey on Synthetic Infrared Image synthesis
by: Upadhyay, Avinash, et al.
Published: (2024)

DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer
by: Huang, Haiduo, et al.
Published: (2025)