Image de couverture de livre

Forget, Then Recall: Learnable Compression and Selective Unfolding via Gist Sparse Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Mao, Yuzhen, Li, Michael Y., Fox, Emily B.
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Long Context In-Context Compression by Getting to the Gist of Gisting
par: Petrov, Aleksandar, et autres
Publié: (2025)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
par: He, Mutian, et autres
Publié: (2025)

Neural Garbage Collection: Learning to Forget while Learning to Reason
par: Li, Michael Y., et autres
Publié: (2026)

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
par: Piękos, Piotr, et autres
Publié: (2025)

Sparse graphs using exchangeable random measures
par: Caron, François, et autres
Publié: (2014)

SALS: Sparse Attention in Latent Space for KV cache Compression
par: Mu, Junlin, et autres
Publié: (2025)

SLA2: Sparse-Linear Attention with Learnable Routing and QAT
par: Zhang, Jintao, et autres
Publié: (2026)

Automated Statistical Model Discovery with Language Models
par: Li, Michael Y., et autres
Publié: (2024)

Tuning-Free Structured Sparse PCA via Deep Unfolding Networks
par: Chen, Long, et autres
Publié: (2025)

IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs
par: Mao, Yuzhen, et autres
Publié: (2024)

Representation Unlearning: Forgetting through Information Compression
par: Almudévar, Antonio, et autres
Publié: (2026)

Learnability in Online Kernel Selection with Memory Constraint via Data-dependent Regret Analysis
par: Li, Junfan, et autres
Publié: (2024)

Gated Graph Attention Networks with Learnable Temperature
par: Ma, Zhongtian, et autres
Publié: (2026)

A Two-Phase Recall-and-Select Framework for Fast Model Selection
par: Cui, Jianwei, et autres
Publié: (2024)

Reflecting Topology Consistency and Abnormality via Learnable Attentions for Airway Labeling
par: Li, Chenyu, et autres
Publié: (2024)

Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearning
par: Shang, Bingqi, et autres
Publié: (2025)

Forgetting Transformer: Softmax Attention with a Forget Gate
par: Lin, Zhixuan, et autres
Publié: (2025)

Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression
par: Gao, Junqi, et autres
Publié: (2026)

Domain Adaptation and Multi-view Attention for Learnable Landmark Tracking with Sparse Data
par: Chase Jr, Timothy, et autres
Publié: (2025)

BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis
par: Ni, Zelin, et autres
Publié: (2023)

Robust Learnability of Sample-Compressible Distributions under Noisy or Adversarial Perturbations
par: Boushehrian, Arefe, et autres
Publié: (2025)

Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys
par: Yang, Xu, et autres
Publié: (2026)

ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression
par: Liu, Guangda, et autres
Publié: (2024)

Learnable Sparse Customization in Heterogeneous Edge Computing
par: Xue, Jingjing, et autres
Publié: (2024)

OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?
par: Jiang, Liangze, et autres
Publié: (2024)

CriticAL: Critic Automation with Language Models
par: Li, Michael Y., et autres
Publié: (2024)

Learning to Forget: Bayesian Time Series Forecasting using Recurrent Sparse Spectrum Signature Gaussian Processes
par: Tóth, Csaba, et autres
Publié: (2024)

Learning to Forget Attention: Memory Consolidation for Adaptive Compute Reduction
par: Shihab, Ibne Farabi, et autres
Publié: (2026)

Solving Sparse \& High-Dimensional-Output Regression via Compression
par: Li, Renyuan, et autres
Publié: (2024)

Understanding Catastrophic Forgetting In LoRA via Mean-Field Attention Dynamics
par: Koubbi, Hugo, et autres
Publié: (2024)

Through a Compressed Lens: Investigating The Impact of Quantization on Factual Knowledge Recall
par: Wang, Qianli, et autres
Publié: (2025)

Semi-Unified Sparse Dictionary Learning with Learnable Top-K LISTA and FISTA Encoders
par: Lin, Fengsheng, et autres
Publié: (2025)

Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning
par: Khoriaty, Matthew, et autres
Publié: (2025)

Vectorized Attention with Learnable Encoding for Quantum Transformer
par: Guo, Ziqing, et autres
Publié: (2025)

Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers
par: Chen, Anrui, et autres
Publié: (2026)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
par: Gao, Yizhao, et autres
Publié: (2025)

Forget the Data and Fine-Tuning! Just Fold the Network to Compress
par: Wang, Dong, et autres
Publié: (2025)

Experimental Analysis of Large-scale Learnable Vector Storage Compression
par: Zhang, Hailin, et autres
Publié: (2023)

IceCache: Memory-efficient KV-cache Management for Long-Sequence LLMs
par: Mao, Yuzhen, et autres
Publié: (2026)

SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
par: S, Santhosh G, et autres
Publié: (2025)