Cover Image

Rethinking Momentum Knowledge Distillation in Online Continual Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Michel, Nicolas, Wang, Maorong, Xiao, Ling, Yamasaki, Toshihiko
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradients
by: Michel, Nicolas, et al.
Published: (2025)

Improving Plasticity in Online Continual Learning via Collaborative Learning
by: Wang, Maorong, et al.
Published: (2023)

Dealing with Synthetic Data Contamination in Online Continual Learning
by: Wang, Maorong, et al.
Published: (2024)

Continual Distillation of Teachers from Different Domains
by: Michel, Nicolas, et al.
Published: (2026)

Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval
by: Xiao, Ling, et al.
Published: (2022)

A Multihead Continual Learning Framework for Fine-Grained Fashion Image Retrieval with Contrastive Learning and Exponential Moving Average Distillation
by: Xiao, Ling, et al.
Published: (2026)

Reward Incremental Learning in Text-to-Image Generation
by: Wang, Maorong, et al.
Published: (2024)

Teach Harder, Learn Poorer: Rethinking Hard Sample Distillation for GNN-to-MLP Knowledge Distillation
by: Wu, Lirong, et al.
Published: (2024)

Online Adversarial Knowledge Distillation for Graph Neural Networks
by: Wang, Can, et al.
Published: (2021)

LLM-Advisor: An LLM Benchmark for Cost-efficient Path Planning across Multiple Terrains
by: Xiao, Ling, et al.
Published: (2025)

Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning
by: Gao, Haozhi, et al.
Published: (2024)

Rethinking the Foundations for Continual Reinforcement Learning
by: Elelimy, Esraa, et al.
Published: (2025)

Low-redundancy Distillation for Continual Learning
by: Liu, RuiQi, et al.
Published: (2023)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

Taming Momentum: Rethinking Optimizer States Through Low-Rank Approximation
by: Wang, Zhengbo, et al.
Published: (2026)

DistilCLIP-EEG: Enhancing Epileptic Seizure Detection Through Multi-modal Learning and Knowledge Distillation
by: Wang, Zexin, et al.
Published: (2025)

Rethinking the Role of Temperature in Large Language Model Distillation
by: Luong, Hoang-Chau, et al.
Published: (2026)

Balancing Knowledge Distillation for Imbalance Learning with Bilevel Optimization
by: Nguyen, Anh B. H., et al.
Published: (2026)

Knowledge Distillation Must Account for What It Loses
by: Wang, Wenshuo
Published: (2026)

Learning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer
by: Dehigahawattage, Nilushika Udayangani Hewa, et al.
Published: (2026)

Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers
by: Miyai, Atsuyuki, et al.
Published: (2026)

Online Policy Distillation with Decision-Attention
by: Yu, Xinqiang, et al.
Published: (2024)

Coupled Distributional Random Expert Distillation for World Model Online Imitation Learning
by: Li, Shangzhe, et al.
Published: (2025)

Multi-Level Knowledge Distillation and Dynamic Self-Supervised Learning for Continual Learning
by: Kim, Taeheon, et al.
Published: (2025)

Distribution-aware Online Continual Learning for Urban Spatio-Temporal Forecasting
by: Wang, Chengxin, et al.
Published: (2024)

PolyGen: Fully Synthetic Vision-Language Training via Multi-Generator Ensembles
by: Brusini, Leonardo, et al.
Published: (2026)

Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systems
by: Khani, Nikhil, et al.
Published: (2024)

Hierarchically Gated Experts for Efficient Online Continual Learning
by: Luong, Kevin, et al.
Published: (2024)

Continual Reinforcement Learning by Planning with Online World Models
by: Liu, Zichen, et al.
Published: (2025)

Federated Continual Learning via Knowledge Fusion: A Survey
by: Yang, Xin, et al.
Published: (2023)

Robust Knowledge Distillation Based on Feature Variance Against Backdoored Teacher Model
by: Chen, Jinyin, et al.
Published: (2024)

Right Time to Learn:Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillation
by: Sun, Guanglong, et al.
Published: (2025)

Graph Knowledge Distillation to Mixture of Experts
by: Rumiantsev, Pavel, et al.
Published: (2024)

Dynamic Temperature Scheduler for Knowledge Distillation
by: Islam, Sibgat Ul, et al.
Published: (2025)

Membership and Memorization in LLM Knowledge Distillation
by: Zhang, Ziqi, et al.
Published: (2025)

Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments
by: Meng, Guanlin
Published: (2023)

Group Relative Knowledge Distillation: Learning from Teacher's Relational Inductive Bias
by: Li, Chao, et al.
Published: (2025)

S^2-KD: Semantic-Spectral Knowledge Distillation Spatiotemporal Forecasting
by: Wang, Wenshuo, et al.
Published: (2025)

Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning
by: Ahn, Hongjoon, et al.
Published: (2024)

Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation
by: Yan, HongWei, et al.
Published: (2024)