Imagen de Portada

How to Train the Teacher Model for Effective Knowledge Distillation

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Hamidi, Shayan Mohajer, Deng, Xizhen, Tan, Renhao, Ye, Linfeng, Salamah, Ahmed Hussein
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information
por: Ye, Linfeng, et al.
Publicado: (2024)

Distributed Quasi-Newton Method for Fair and Fast Federated Learning
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2025)

Robustness Against Adversarial Attacks via Learning Confined Adversarial Polytopes
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Thundernna: a white box adversarial attack
por: Ye, Linfeng, et al.
Publicado: (2021)

Towards Undistillable Models by Minimizing Conditional Mutual Information
por: Ye, Linfeng, et al.
Publicado: (2025)

Rate-Constrained Quantization for Communication-Efficient Federated Learning
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Coded Deep Learning: Framework and Algorithm
por: Yang, En-hui, et al.
Publicado: (2025)

Conditional Mutual Information Based Diffusion Posterior Sampling for Solving Inverse Problems
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2025)

Information-Guided Diffusion Sampling for Dataset Distillation
por: Ye, Linfeng, et al.
Publicado: (2025)

AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Regime-aware financial volatility forecasting via in-context learning
por: Asaad, Saba, et al.
Publicado: (2026)

Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Sampling
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2025)

GP-FL: Model-Based Hessian Estimation for Second-Order Over-the-Air Federated Learning
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Over-the-Air Fair Federated Learning via Multi-Objective Optimization
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2025)

Practical Insights into Knowledge Distillation for Pre-Trained Models
por: Alballa, Norah, et al.
Publicado: (2024)

Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling
por: Hamidi, Shayan Mohajer, et al.
Publicado: (2024)

Toward Student-Oriented Teacher Network Training For Knowledge Distillation
por: Dong, Chengyu, et al.
Publicado: (2022)

Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
por: Binici, Kuluhan, et al.
Publicado: (2024)

Model Merging via Multi-Teacher Knowledge Distillation
por: Dalili, Seyed Arshan, et al.
Publicado: (2025)

A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation
por: Wu, Lirong, et al.
Publicado: (2024)

How Is Uncertainty Propagated in Knowledge Distillation?
por: Cui, Ziyao, et al.
Publicado: (2026)

Knowledge Distillation with Training Wheels
por: Liu, Guanlin, et al.
Publicado: (2025)

SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines
por: Morad, Itai, et al.
Publicado: (2026)

Dual-Forward Path Teacher Knowledge Distillation: Bridging the Capacity Gap Between Teacher and Student
por: Li, Tong, et al.
Publicado: (2025)

In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
por: Panigrahi, Abhishek, et al.
Publicado: (2025)

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
por: Fang, Luyang, et al.
Publicado: (2026)

Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation
por: Hu, Chengming, et al.
Publicado: (2023)

Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Model
por: Chen, Jinyin, et al.
Publicado: (2024)

Task-Oriented GNNs Training on Large Knowledge Graphs for Accurate and Efficient Modeling
por: Abdallah, Hussein, et al.
Publicado: (2024)

Knowledge Distillation Based on Transformed Teacher Matching
por: Zheng, Kaixiang, et al.
Publicado: (2024)

Knowledge Distillation with Adapted Weight
por: Wu, Sirong, et al.
Publicado: (2025)

The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms
por: Bayati, Mohsen, et al.
Publicado: (2020)

The Role of Teacher Calibration in Knowledge Distillation
por: Kim, Suyoung, et al.
Publicado: (2025)

How to Backdoor the Knowledge Distillation
por: Wu, Chen, et al.
Publicado: (2025)

Adversarial Domain Adaptation Enables Knowledge Transfer Across Heterogeneous RNA-Seq Datasets
por: Dradjat, Kevin, et al.
Publicado: (2026)

FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
por: Shadin, Nazmus Shakib, et al.
Publicado: (2026)

LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations
por: Vujanic, Robin, et al.
Publicado: (2025)

Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation
por: Aslam, Muhammad Haseeb, et al.
Publicado: (2025)

HPM-KD: Hierarchical Progressive Multi-Teacher Framework for Knowledge Distillation and Efficient Model Compression
por: Haase, Gustavo Coelho, et al.
Publicado: (2025)