Imagen de Portada

Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Niu, Wenqi, Wang, Yingchao, Cai, Guohui, Hou, Hanpo
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Model
por: Chen, Jinyin, et al.
Publicado: (2024)

Federated Progressive Self-Distillation with Logits Calibration for Personalized IIoT Edge Intelligence
por: Wang, Yingchao, et al.
Publicado: (2024)

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
por: Fang, Luyang, et al.
Publicado: (2026)

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
por: Nair, Lakshmi
Publicado: (2024)

Model Merging via Multi-Teacher Knowledge Distillation
por: Dalili, Seyed Arshan, et al.
Publicado: (2025)

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay
por: Binici, Kuluhan, et al.
Publicado: (2022)

DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher
por: Zhong, Yisheng, et al.
Publicado: (2026)

Toward Student-Oriented Teacher Network Training For Knowledge Distillation
por: Dong, Chengyu, et al.
Publicado: (2022)

Group Relative Knowledge Distillation: Learning from Teacher's Relational Inductive Bias
por: Li, Chao, et al.
Publicado: (2025)

GSTAM: Efficient Graph Distillation with Structural Attention-Matching
por: Rasti-Meymandi, Arash, et al.
Publicado: (2024)

Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
por: Binici, Kuluhan, et al.
Publicado: (2024)

FedMTFI: Feature Importance Based Optimized Multi Teacher Knowledge Distillation in Heterogeneous Federated Learning Environment
por: Shadin, Nazmus Shakib, et al.
Publicado: (2026)

Membership and Memorization in LLM Knowledge Distillation
por: Zhang, Ziqi, et al.
Publicado: (2025)

Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher
por: Ji, Guangda, et al.
Publicado: (2020)

The Role of Teacher Calibration in Knowledge Distillation
por: Kim, Suyoung, et al.
Publicado: (2025)

Distilling Genomic Models for Efficient mRNA Representation Learning via Embedding Matching
por: Haidari, Rasched, et al.
Publicado: (2026)

Geometric Flow Matching for Molecular Conformation Generation via Manifold Decomposition
por: Liu, Yunqing, et al.
Publicado: (2026)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
por: Monsefi, Amin Karimi, et al.
Publicado: (2026)

DistDD: Distributed Data Distillation Aggregation through Gradient Matching
por: Wang, Peiran, et al.
Publicado: (2024)

Preference-Based Self-Distillation: Beyond KL Matching via Reward Regularization
por: Yu, Xin, et al.
Publicado: (2026)

Knowledge Distillation Must Account for What It Loses
por: Wang, Wenshuo
Publicado: (2026)

LLMBoost: Make Large Language Models Stronger with Boosting
por: Chen, Zehao, et al.
Publicado: (2025)

Path-Guided Flow Matching for Dataset Distillation
por: Li, Xuhui, et al.
Publicado: (2026)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
por: Tian, Yijun, et al.
Publicado: (2024)

Efficient Knowledge Distillation via Curriculum Extraction
por: Gupta, Shivam, et al.
Publicado: (2025)

Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
por: Park, Seonghyeon, et al.
Publicado: (2026)

Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments
por: Meng, Guanlin
Publicado: (2023)

Good Teachers Explain: Explanation-Enhanced Knowledge Distillation
por: Parchami-Araghi, Amin, et al.
Publicado: (2024)

Teacher as a Lenient Expert: Teacher-Agnostic Data-Free Knowledge Distillation
por: Shin, Hyunjune, et al.
Publicado: (2024)

Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalization
por: Chuang, Yu-Neng, et al.
Publicado: (2025)

Dimer-Enhanced Optimization: A First-Order Approach to Escaping Saddle Points in Neural Network Training
por: Hu, Yue, et al.
Publicado: (2025)

Multi-Stage Knowledge-Distilled VGAE and GAT for Robust Controller-Area-Network Intrusion Detection
por: Frenken, Robert, et al.
Publicado: (2025)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
por: Sun, Mengyang, et al.
Publicado: (2025)

Online Adversarial Knowledge Distillation for Graph Neural Networks
por: Wang, Can, et al.
Publicado: (2021)

Linear Projections of Teacher Embeddings for Few-Class Distillation
por: Loo, Noel, et al.
Publicado: (2024)

Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation
por: Wu, Yecheng, et al.
Publicado: (2026)

Teaching the Teacher: The Role of Teacher-Student Smoothness Alignment in Genetic Programming-based Symbolic Distillation
por: Dhar, Soumyadeep, et al.
Publicado: (2025)

Rethinking Momentum Knowledge Distillation in Online Continual Learning
por: Michel, Nicolas, et al.
Publicado: (2023)

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
por: Liu, Xiaogeng, et al.
Publicado: (2026)

Enhancing Transformer with GNN Structural Knowledge via Distillation: A Novel Approach
por: Duan, Zhihua, et al.
Publicado: (2025)