Cover Image

Rethinking Decoupled Knowledge Distillation: A Predictive Distribution Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Bowen, Cheng, Ran
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment
by: Yu, Chengting, et al.
Published: (2024)

Rethinking Score Distillation as a Bridge Between Image Distributions
by: McAllister, David, et al.
Published: (2024)

DeepKD: A Deeply Decoupled and Denoised Knowledge Distillation Trainer
by: Huang, Haiduo, et al.
Published: (2025)

Knowledge Distillation Based on Transformed Teacher Matching
by: Zheng, Kaixiang, et al.
Published: (2024)

Rethinking Autoencoders for Medical Anomaly Detection from A Theoretical Perspective
by: Cai, Yu, et al.
Published: (2024)

Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition
by: Guo, Guangyu, et al.
Published: (2021)

Distilling Knowledge for Short-to-Long Term Trajectory Prediction
by: Das, Sourav, et al.
Published: (2023)

Multi-level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL
by: Su, Shibin, et al.
Published: (2025)

Rethinking Dataset Distillation: Hard Truths about Soft Labels
by: Dey, Priyam, et al.
Published: (2026)

Generative Distribution Distillation
by: Cui, Jiequan, et al.
Published: (2025)

Distilling the Knowledge in Data Pruning
by: Ben-Baruch, Emanuel, et al.
Published: (2024)

Rethinking the Diffusion Model from a Langevin Perspective
by: Zheng, Candi, et al.
Published: (2026)

Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning
by: Zhang, Qifan, et al.
Published: (2024)

Rethinking Timesteps Samplers and Prediction Types
by: Xie, Bin, et al.
Published: (2025)

Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective
by: Wen, Qishuai, et al.
Published: (2024)

Dataset Distillation as Data Compression: A Rate-Utility Perspective
by: Bao, Youneng, et al.
Published: (2025)

Why does Knowledge Distillation Work? Rethink its Attention and Fidelity Mechanism
by: Guo, Chenqi, et al.
Published: (2024)

Data-Augmented Quantization-Aware Knowledge Distillation
by: Kur, Justin, et al.
Published: (2025)

Improving Neural ODEs via Knowledge Distillation
by: Chu, Haoyu, et al.
Published: (2022)

Densely Distilling Cumulative Knowledge for Continual Learning
by: Shi, Zenglin, et al.
Published: (2024)

Forward-Backward Knowledge Distillation for Continual Clustering
by: Sadeghi, Mohammadreza, et al.
Published: (2024)

Adaptive Teaching with Shared Classifier for Knowledge Distillation
by: Jang, Jaeyeon, et al.
Published: (2024)

A Unified Revisit of Temperature in Classification-Based Knowledge Distillation
by: Frank, Logan, et al.
Published: (2026)

DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
by: Li, Quanhao, et al.
Published: (2026)

PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors
by: Moser, Brian B., et al.
Published: (2025)

Enriching Knowledge Distillation with Intra-Class Contrastive Learning
by: Yuan, Hua, et al.
Published: (2025)

Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation
by: Huang, Yanglin, et al.
Published: (2025)

Adaptive Temperature Based on Logits Correlation in Knowledge Distillation
by: Matsuyama, Kazuhiro, et al.
Published: (2025)

Preview-based Category Contrastive Learning for Knowledge Distillation
by: Ding, Muhe, et al.
Published: (2024)

Prototypical Distillation and Debiased Tuning for Black-box Unsupervised Domain Adaptation
by: Liang, Jian, et al.
Published: (2024)

Rethinking the Bias of Foundation Model under Long-tailed Distribution
by: Chen, Jiahao, et al.
Published: (2025)

Topology-Guided Knowledge Distillation for Efficient Point Cloud Processing
by: Hai, Luu Tung, et al.
Published: (2025)

Conditional Pseudo-Supervised Contrast for Data-Free Knowledge Distillation
by: Shao, Renrong, et al.
Published: (2025)

The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
by: Son, Seungwoo, et al.
Published: (2023)

On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning
by: Fang, Pengfei, et al.
Published: (2024)

Highlight Every Step: Knowledge Distillation via Collaborative Teaching
by: Zhao, Haoran, et al.
Published: (2019)

Improving Diversity in Black-box Few-shot Knowledge Distillation
by: Vo, Tri-Nhan, et al.
Published: (2026)

Domain Generalization for Crop Segmentation with Standardized Ensemble Knowledge Distillation
by: Angarano, Simone, et al.
Published: (2023)

ReffAKD: Resource-efficient Autoencoder-based Knowledge Distillation
by: Doshi, Divyang, et al.
Published: (2024)

MST-Distill: Mixture of Specialized Teachers for Cross-Modal Knowledge Distillation
by: Li, Hui, et al.
Published: (2025)