Cover Image

Linear Projections of Teacher Embeddings for Few-Class Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Loo, Noel, Iliopoulos, Fotis, Hu, Wei, Vee, Erik
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GUIDE: Guided Initialization and Distillation of Embeddings
by: Trinh, Khoa, et al.
Published: (2025)

CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers
by: Nair, Lakshmi
Published: (2024)

Few-Shot Class-Incremental Learning with Prior Knowledge
by: Jiang, Wenhao, et al.
Published: (2024)

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
by: Monsefi, Amin Karimi, et al.
Published: (2026)

Few-shot Class-incremental Learning for Classification and Object Detection: A Survey
by: Zhang, Jinghua, et al.
Published: (2023)

NTK-Guided Few-Shot Class Incremental Learning
by: Liu, Jingren, et al.
Published: (2024)

An experimental approach on Few Shot Class Incremental Learning
by: Adam, Marinela
Published: (2025)

Few-shot Class-incremental Fault Diagnosis by Preserving Class-Agnostic Knowledge with Dual-Granularity Representations
by: Yang, Zhendong, et al.
Published: (2025)

Adaptive Locally Linear Embedding
by: Goli, Ali, et al.
Published: (2025)

Model Merging via Multi-Teacher Knowledge Distillation
by: Dalili, Seyed Arshan, et al.
Published: (2025)

Few-Shot Class-Incremental Learning with Non-IID Decentralized Data
by: Liu, Cuiwei, et al.
Published: (2024)

Few-Shot Class Incremental Learning via Robust Transformer Approach
by: Paeedeh, Naeem, et al.
Published: (2024)

An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning
by: Li, Dong, et al.
Published: (2024)

Teaching the Teacher: The Role of Teacher-Student Smoothness Alignment in Genetic Programming-based Symbolic Distillation
by: Dhar, Soumyadeep, et al.
Published: (2025)

Multi-Teacher Knowledge Distillation via Teacher-Informed Mixture Priors
by: Fang, Luyang, et al.
Published: (2026)

DUET: Distilled LLM Unlearning from an Efficiently Contextualized Teacher
by: Zhong, Yisheng, et al.
Published: (2026)

Toward Student-Oriented Teacher Network Training For Knowledge Distillation
by: Dong, Chengyu, et al.
Published: (2022)

From Teacher to Student: Tracking Memorization Through Model Distillation
by: Singh, Simardeep
Published: (2025)

Distilling Linearized Behavior into Non-Linear Fine-Tuning for Effective Task Arithmetic
by: Sommariva, Thomas, et al.
Published: (2026)

TabDistill: Distilling Transformers into Neural Nets for Few-Shot Tabular Classification
by: Dissanayake, Pasan, et al.
Published: (2025)

Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
by: Binici, Kuluhan, et al.
Published: (2024)

Heuristic Methods are Good Teachers to Distill MLPs for Graph Link Prediction
by: Qin, Zongyue, et al.
Published: (2025)

Aggregated f-average Neural Network applied to Few-Shot Class Incremental Learning
by: Vu, Mathieu, et al.
Published: (2023)

On Teacher Hacking in Language Model Distillation
by: Tiapkin, Daniil, et al.
Published: (2025)

Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation
by: Zhang, Xiaoxiong, et al.
Published: (2024)

Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Model
by: Chen, Jinyin, et al.
Published: (2024)

When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
by: Liu, Xiaogeng, et al.
Published: (2026)

Group Relative Knowledge Distillation: Learning from Teacher's Relational Inductive Bias
by: Li, Chao, et al.
Published: (2025)

Linear Bellman Completeness Suffices for Efficient Online Reinforcement Learning with Few Actions
by: Golowich, Noah, et al.
Published: (2024)

Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning
by: Hou, Iyad Ait, et al.
Published: (2026)

Beyond Linearity in Attention Projections: The Case for Nonlinear Queries
by: Karbevski, Marko
Published: (2026)

Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching
by: Niu, Wenqi, et al.
Published: (2024)

Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher
by: Ji, Guangda, et al.
Published: (2020)

Does Prior Data Matter? Exploring Joint Training in the Context of Few-Shot Class-Incremental Learning
by: Kim, Shiwon, et al.
Published: (2025)

eCIL-MU: Embedding based Class Incremental Learning and Machine Unlearning
by: Zuo, Zhiwei, et al.
Published: (2024)

Linear Dimensionality Reduction for Word Embeddings in Tabular Data Classification
by: Ressel, Liam, et al.
Published: (2025)

Distilling Genomic Models for Efficient mRNA Representation Learning via Embedding Matching
by: Haidari, Rasched, et al.
Published: (2026)

Continuous-Time Linear Positional Embedding for Irregular Time Series Forecasting
by: Kim, Byunghyun, et al.
Published: (2024)

Learning with Embedded Linear Equality Constraints via Variational Bayesian Inference
by: Marsh, Matthew, et al.
Published: (2026)

Class Incremental Fault Diagnosis under Limited Fault Data via Supervised Contrastive Knowledge Distillation
by: Zhang, Hanrong, et al.
Published: (2025)