Cover Image

Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xue, Huiyan, Ran, Xuming, Li, Yaxin, Xu, Qi, Li, Enhui, Xu, Yi, Zhang, Qiang
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Representation Finetuning for Continual Learning
by: Luo, Haihua, et al.
Published: (2026)

Beyond Distribution Estimation: Simplex Anchored Structural Inference Towards Universal Semi-Supervised Learning
by: Hou, Yaxin, et al.
Published: (2026)

SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
by: Zhang, Mingxu, et al.
Published: (2026)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

Efficient ANN-SNN Conversion with Error Compensation Learning
by: Liu, Chang, et al.
Published: (2025)

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
by: Xu, Yuanda, et al.
Published: (2026)

Context Distillation as Latent Memory Management
by: Zheng, Ziyang, et al.
Published: (2026)

Privacy-Preserving Transfer Learning for Community Detection using Locally Distributed Multiple Networks
by: Guo, Xiao, et al.
Published: (2025)

Low-redundancy Distillation for Continual Learning
by: Liu, RuiQi, et al.
Published: (2023)

Continual Policy Distillation from Distributed Reinforcement Learning Teachers
by: Li, Yuxuan, et al.
Published: (2026)

PDAC: Efficient Coreset Selection for Continual Learning via Probability Density Awareness
by: Gao, Junqi, et al.
Published: (2025)

Beyond Tokens: Enhancing RTL Quality Estimation via Structural Graph Learning
by: Liu, Yi, et al.
Published: (2025)

Attribution-Guided Distillation of Matryoshka Sparse Autoencoders
by: Martin-Linares, Cristina P., et al.
Published: (2025)

HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
by: Fang, Yuan, et al.
Published: (2026)

De-confounding Representation Learning for Counterfactual Inference on Continuous Treatment via Generative Adversarial Network
by: Zhao, Yonghe, et al.
Published: (2023)

Memory-Statistics Tradeoff in Continual Learning with Structural Regularization
by: Li, Haoran, et al.
Published: (2025)

Generative Distribution Distillation
by: Cui, Jiequan, et al.
Published: (2025)

Distribution-Level Memory Recall for Continual Learning: Preserving Knowledge and Avoiding Confusion
by: Cheng, Shaoxu, et al.
Published: (2024)

Continual Contrastive Learning on Tabular Data with Out of Distribution
by: Ginanjar, Achmad, et al.
Published: (2025)

Vector Quantization Prompting for Continual Learning
by: Jiao, Li, et al.
Published: (2024)

Ultra-Sparse Memory Network
by: Huang, Zihao, et al.
Published: (2024)

MGSER-SAM: Memory-Guided Soft Experience Replay with Sharpness-Aware Optimization for Enhanced Continual Learning
by: Li, Xingyu, et al.
Published: (2024)

Global Momentum Compression for Sparse Communication in Distributed Learning
by: Shi, Chang-Wei, et al.
Published: (2019)

Operator-Guided Invariance Learning for Continuous Reinforcement Learning
by: Zhang, Zuyuan, et al.
Published: (2026)

Pushing the Limits of Distillation-Based Continual Learning via Classifier-Proximal Lightweight Plugins
by: Xu, Zhiming, et al.
Published: (2025)

Structure Learning with Continuous Optimization: A Sober Look and Beyond
by: Ng, Ignavier, et al.
Published: (2023)

Byzantine-Robust Distributed Sparse Learning Revisited
by: Wang, Yuxuan, et al.
Published: (2026)

Brain-inspired continual pre-trained learner via silent synaptic consolidation
by: Ran, Xuming, et al.
Published: (2024)

Path-Guided Flow Matching for Dataset Distillation
by: Li, Xuhui, et al.
Published: (2026)

Energy-Structured Low-Rank Adaptation for Continual Learning
by: Li, Longhua, et al.
Published: (2026)

Attribution-Guided Continual Learning for Large Language Models
by: Liu, Yazheng, et al.
Published: (2026)

Group-Sparse Matrix Factorization for Transfer Learning of Word Embeddings
by: Xu, Kan, et al.
Published: (2021)

Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation
by: Li, Jiyong, et al.
Published: (2024)

Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
by: Muhamed, Aashiq, et al.
Published: (2024)

Exploration by Random Distribution Distillation
by: Fang, Zhirui, et al.
Published: (2025)

Enhancing Pretrained Model-based Continual Representation Learning via Guided Random Projection
by: Li, Ruilin, et al.
Published: (2026)

Random Walk Guided Hyperbolic Graph Distillation
by: Long, Yunbo, et al.
Published: (2025)

Feature Distillation is the Better Choice for Model-Heterogeneous Federated Learning
by: Li, Yichen, et al.
Published: (2025)

Improving Knowledge Distillation in Transfer Learning with Layer-wise Learning Rates
by: Kokane, Shirley, et al.
Published: (2024)

Information-Theoretic Dual Memory System for Continual Learning
by: Wu, RunQing, et al.
Published: (2025)