Cover Image

ScaleKD: Strong Vision Transformers Could Be Excellent Teachers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fan, Jiawei, Li, Chao, Liu, Xiaolong, Yao, Anbang
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Chain-of-Models Pre-Training: Rethinking Training Acceleration of Vision Foundation Models
by: Fan, Jiawei, et al.
Published: (2026)

KernelWarehouse: Rethinking the Design of Dynamic Convolution
by: Li, Chao, et al.
Published: (2024)

Small Scale Data-Free Knowledge Distillation
by: Liu, He, et al.
Published: (2024)

Morse: Dual-Sampling for Lossless Acceleration of Diffusion Models
by: Li, Chao, et al.
Published: (2025)

HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction
by: Yuan, Ruicheng, et al.
Published: (2026)

SAFE-KD: Risk-Controlled Early-Exit Distillation for Vision Backbones
by: Khazem, Salim
Published: (2026)

Mechanisms of Non-Monotonic Scaling in Vision Transformers
by: Kumar, Anantha Padmanaban Krishna
Published: (2025)

Fairness-aware Vision Transformer via Debiased Self-Attention
by: Qiang, Yao, et al.
Published: (2023)

Scaling Semantic Categories: Investigating the Impact on Vision Transformer Labeling Performance
by: Lamelas, Anthony, et al.
Published: (2025)

Improving Interpretation Faithfulness for Vision Transformers
by: Hu, Lijie, et al.
Published: (2023)

Discovering Influential Neuron Path in Vision Transformers
by: Wang, Yifan, et al.
Published: (2025)

DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer
by: Huang, Haiduo, et al.
Published: (2025)

Geometric-Guided Few-Shot Dental Landmark Detection with Human-Centric Foundation Model
by: Wang, Anbang, et al.
Published: (2025)

EEO-TFV: Escape-Explore Optimizer for Web-Scale Time-Series Forecasting and Vision Analysis
by: Wang, Hua, et al.
Published: (2026)

KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
by: Nourbakhsh, Erfan, et al.
Published: (2025)

DiffiT: Diffusion Vision Transformers for Image Generation
by: Hatamizadeh, Ali, et al.
Published: (2023)

Intriguing Equivalence Structures of the Embedding Space of Vision Transformers
by: Salman, Shaeke, et al.
Published: (2024)

Masking Teacher and Reinforcing Student for Distilling Vision-Language Models
by: Lee, Byung-Kwan, et al.
Published: (2025)

Stratified Knowledge-Density Super-Network for Scalable Vision Transformers
by: Li, Longhua, et al.
Published: (2025)

NOAH: Learning Pairwise Object Category Attentions for Image Classification
by: Li, Chao, et al.
Published: (2024)

On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality
by: Hu, Jerry Yao-Chieh, et al.
Published: (2024)

Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling
by: Mekonnen, Kidist Amde, et al.
Published: (2024)

Block-Recurrent Dynamics in Vision Transformers
by: Jacobs, Mozes, et al.
Published: (2025)

DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learning
by: Ienco, Dino, et al.
Published: (2024)

Transferable Model-agnostic Vision-Language Model Adaptation for Efficient Weak-to-Strong Generalization
by: Park, Jihwan, et al.
Published: (2025)

The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design
by: Liu, Anjie, et al.
Published: (2026)

Scaling Diffusion Transformers Efficiently via $μ$P
by: Zheng, Chenyu, et al.
Published: (2025)

VariViT: A Vision Transformer for Variable Image Sizes
by: Varma, Aswathi, et al.
Published: (2026)

Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applications
by: Hu, Zixuan, et al.
Published: (2025)

Continual Adaptation of Vision Transformers for Federated Learning
by: Halbe, Shaunak, et al.
Published: (2023)

ADAPT to Robustify Prompt Tuning Vision Transformers
by: Eskandar, Masih, et al.
Published: (2024)

Accelerating Vision Transformers with Adaptive Patch Sizes
by: Choudhury, Rohan, et al.
Published: (2025)

Class-Discriminative Attention Maps for Vision Transformers
by: Brocki, Lennart, et al.
Published: (2023)

Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding
by: Fang, Yixiong, et al.
Published: (2024)

Gaze-Informed Vision Transformers: Predicting Driving Decisions Under Uncertainty
by: Koorathota, Sharath, et al.
Published: (2023)

MedSapiens: Taking a Pose to Rethink Medical Imaging Landmark Detection
by: Elbatel, Marawan, et al.
Published: (2025)

VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL
by: Feng, Yichen, et al.
Published: (2025)

SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
by: Ataiefard, Foozhan, et al.
Published: (2024)

Intriguing Differences Between Zero-Shot and Systematic Evaluations of Vision-Language Transformer Models
by: Salman, Shaeke, et al.
Published: (2024)

Oscillation-Reduced MXFP4 Training for Vision Transformers
by: Chen, Yuxiang, et al.
Published: (2025)