Imagen de Portada

Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Ji, Guangda, Zhu, Zhanxing
Formato:	Preprint
Publicado:	2020
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Toward Student-Oriented Teacher Network Training For Knowledge Distillation
por: Dong, Chengyu, et al.
Publicado: (2022)

Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
por: Binici, Kuluhan, et al.
Publicado: (2024)

Neural Latent Arbitrary Lagrangian-Eulerian Grids for Fluid-Solid Interaction
por: Tao, Shilong, et al.
Publicado: (2026)

A Functional Perspective on Knowledge Distillation in Neural Networks
por: Mason-Williams, Israel, et al.
Publicado: (2025)

Online Adversarial Knowledge Distillation for Graph Neural Networks
por: Wang, Can, et al.
Publicado: (2021)

Model Merging via Multi-Teacher Knowledge Distillation
por: Dalili, Seyed Arshan, et al.
Publicado: (2025)

MAVEN: A Mesh-Aware Volumetric Encoding Network for Simulating 3D Flexible Deformation
por: Feng, Zhe, et al.
Publicado: (2026)

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
por: Fang, Luyang, et al.
Publicado: (2026)

Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models
por: Li, Quan, et al.
Publicado: (2024)

Efficient Linear Attention for Multivariate Time Series Modeling via Entropy Equality
por: Zhang, Mingtao, et al.
Publicado: (2025)

Teacher as a Lenient Expert: Teacher-Agnostic Data-Free Knowledge Distillation
por: Shin, Hyunjune, et al.
Publicado: (2024)

Not All Timesteps Matter Equally: Selective Alignment Knowledge Distillation for Spiking Neural Networks
por: Sun, Kai, et al.
Publicado: (2026)

Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Model
por: Chen, Jinyin, et al.
Publicado: (2024)

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
por: Nair, Lakshmi
Publicado: (2024)

Group Relative Knowledge Distillation: Learning from Teacher's Relational Inductive Bias
por: Li, Chao, et al.
Publicado: (2025)

Valid Inference with Imperfect Synthetic Data
por: Byun, Yewon, et al.
Publicado: (2025)

Knowledge Distillation Neural Network for Predicting Car-following Behaviour of Human-driven and Autonomous Vehicles
por: Adewale, Ayobami, et al.
Publicado: (2024)

DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher
por: Zhong, Yisheng, et al.
Publicado: (2026)

Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching
por: Niu, Wenqi, et al.
Publicado: (2024)

Logical Distillation of Graph Neural Networks
por: Pluska, Alexander, et al.
Publicado: (2024)

The Role of Teacher Calibration in Knowledge Distillation
por: Kim, Suyoung, et al.
Publicado: (2025)

OpenGrok: Enhancing SNS Data Processing with Distilled Knowledge and Mask-like Mechanisms
por: AI, Lumen, et al.
Publicado: (2025)

Neural Tangent Knowledge Distillation for Optical Convolutional Networks
por: Xiang, Jinlin, et al.
Publicado: (2025)

Condensed Data Expansion Using Model Inversion for Knowledge Distillation
por: Binici, Kuluhan, et al.
Publicado: (2024)

Improve Knowledge Distillation via Label Revision and Data Selection
por: Lan, Weichao, et al.
Publicado: (2024)

Rapfi: Distilling Efficient Neural Network for the Game of Gomoku
por: Jin, Zhanggen, et al.
Publicado: (2025)

Distilling Symbolic Priors for Concept Learning into Neural Networks
por: Marinescu, Ioana, et al.
Publicado: (2024)

FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
por: Shadin, Nazmus Shakib, et al.
Publicado: (2026)

Post-Pruning Accuracy Recovery via Data-Free Knowledge Distillation
por: Tripurwar, Chinmay, et al.
Publicado: (2025)

HFedCKD: Toward Robust Heterogeneous Federated Learning via Data-free Knowledge Distillation and Two-way Contrast
por: Zheng, Yiting, et al.
Publicado: (2025)

Knowledge Distillation on Spatial-Temporal Graph Convolutional Network for Traffic Prediction
por: Izadi, Mohammad, et al.
Publicado: (2024)

Good Teachers Explain: Explanation-Enhanced Knowledge Distillation
por: Parchami-Araghi, Amin, et al.
Publicado: (2024)

How to Backdoor the Knowledge Distillation
por: Wu, Chen, et al.
Publicado: (2025)

On the Probabilistic Learnability of Compact Neural Network Preimage Bounds
por: Marzari, Luca, et al.
Publicado: (2025)

Neural Network Verification with Branch-and-Bound for General Nonlinearities
por: Shi, Zhouxing, et al.
Publicado: (2024)

Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation
por: Hasan, Mohammad Junayed, et al.
Publicado: (2023)

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay
por: Binici, Kuluhan, et al.
Publicado: (2022)

Linear Projections of Teacher Embeddings for Few-Class Distillation
por: Loo, Noel, et al.
Publicado: (2024)

Enhancing Knowledge Graph Completion with GNN Distillation and Probabilistic Interaction Modeling
por: Wang, Lingzhi, et al.
Publicado: (2025)

Eau De $Q$-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning
por: Vincent, Théo, et al.
Publicado: (2025)