Cover Image

SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	He, Jinghan, Guo, Haiyun, Zhu, Kuan, Zhao, Zihan, Tang, Ming, Wang, Jinqiao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Systematic Outliers in Large Language Models
by: An, Yongqi, et al.
Published: (2025)

Active Continual Learning: On Balancing Knowledge Retention and Learnability
by: Vu, Thuy-Trang, et al.
Published: (2023)

Self-Consolidating Language Models: Continual Knowledge Incorporation from Context
by: Wang, Zekun, et al.
Published: (2026)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
by: Li, Chengye, et al.
Published: (2025)

GUARD: Guided Unlearning and Retention via Data Attribution for Large Language Models
by: Niu, Peizhi, et al.
Published: (2025)

Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding
by: Li, Linyu, et al.
Published: (2025)

Utilizing Large Language Models for Information Extraction from Real Estate Transactions
by: Zhao, Yu, et al.
Published: (2024)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
by: Cao, Jinghan, et al.
Published: (2026)

A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimization
by: Wang, Ziqing, et al.
Published: (2025)

Large Language Models Lack Temporal Awareness of Medical Knowledge
by: Guan, Zihan, et al.
Published: (2026)

Active Zero: Self-Evolving Vision-Language Models through Active Environment Exploration
by: He, Jinghan, et al.
Published: (2026)

GLBench: A Comprehensive Benchmark for Graph with Large Language Models
by: Li, Yuhan, et al.
Published: (2024)

Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
by: Feng, Yujie, et al.
Published: (2025)

Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
by: He, Jinghan, et al.
Published: (2024)

Continual Learning for Large Language Models: A Survey
by: Wu, Tongtong, et al.
Published: (2024)

Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models
by: Huang, Yanwen, et al.
Published: (2025)

Knowledge Editing on Black-box Large Language Models
by: Song, Xiaoshuai, et al.
Published: (2024)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
by: Zhong, Ming, et al.
Published: (2023)

Unlocking Continual Learning Abilities in Language Models
by: Du, Wenyu, et al.
Published: (2024)

Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models
by: Lee, Joseph, et al.
Published: (2024)

Scaling Linear Attention with Sparse State Expansion
by: Pan, Yuqi, et al.
Published: (2025)

AutoGuide: Automated Generation and Selection of Context-Aware Guidelines for Large Language Model Agents
by: Fu, Yao, et al.
Published: (2024)

Diversity-oriented Data Augmentation with Large Language Models
by: Wang, Zaitian, et al.
Published: (2025)

EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge
by: Zhao, Xuyang, et al.
Published: (2024)

Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model
by: Tong, Kai, et al.
Published: (2025)

Active Model Selection for Large Language Models
by: Durmazkeser, Yavuz, et al.
Published: (2025)

Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
by: Hua, Zhenglin, et al.
Published: (2025)

Knowledgeable Agents by Offline Reinforcement Learning from Large Language Model Rollouts
by: Pang, Jing-Cheng, et al.
Published: (2024)

Not All Features Deserve Attention: Graph-Guided Dependency Learning for Tabular Data Generation with Language Models
by: Zhang, Zheyu, et al.
Published: (2025)

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models
by: Cui, Yingqian, et al.
Published: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
by: He, Wei, et al.
Published: (2024)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
by: Yang, Junjie, et al.
Published: (2025)

InfuserKI: Enhancing Large Language Models with Knowledge Graphs via Infuser-Guided Knowledge Integration
by: Wang, Fali, et al.
Published: (2024)

Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation
by: Ren, Yuxin, et al.
Published: (2023)

HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model
by: Guo, Haiyang, et al.
Published: (2025)

ProxyAttn: Guided Sparse Attention via Representative Heads
by: Wang, Yixuan, et al.
Published: (2025)

CORM: Cache Optimization with Recent Message for Large Language Model Inference
by: Dai, Jincheng, et al.
Published: (2024)

Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention
by: Gao, Bin, et al.
Published: (2024)

CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
by: Huang, Weiyu, et al.
Published: (2025)