Image de couverture de livre

Nectar: Neural Estimation of Cached-Token Attention via Regression

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Monteiro, João, Klein, Michal, Ablin, Pierre, Cuturi, Marco
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
par: Saada, Thiziri Nait, et autres
Publié: (2025)

The Geometries of Truth Are Orthogonal Across Tasks
par: Azizian, Waiss, et autres
Publié: (2025)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
par: Bethune, Louis, et autres
Publié: (2025)

Locking Pretrained Weights via Deep Low-Rank Residual Distillation
par: Sakamoto, Keitaro, et autres
Publié: (2026)

DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures
par: Gualdoni, Eleonora, et autres
Publié: (2026)

Careful with that Scalpel: Improving Gradient Surgery with an EMA
par: Hsieh, Yu-Guan, et autres
Publié: (2024)

Amortizing Maximum Inner Product Search with Learned Support Functions
par: Olausson, Theo X., et autres
Publié: (2026)

Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
par: Ablin, Pierre, et autres
Publié: (2025)

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
par: Grangier, David, et autres
Publié: (2024)

Dynamic Gradient Alignment for Online Data Mixing
par: Fan, Simin, et autres
Publié: (2024)

CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction
par: Goel, Raghavv, et autres
Publié: (2025)

Learning Elastic Costs to Shape Monge Displacements
par: Klein, Michal, et autres
Publié: (2023)

AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
par: Song, Dinghong, et autres
Publié: (2025)

Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing
par: Filippova, Anastasiia, et autres
Publié: (2026)

Scaling Laws for Mixture Pretraining Under Data Constraints
par: Sedova, Anastasiia, et autres
Publié: (2026)

Need a Small Specialized Language Model? Plan Early!
par: Grangier, David, et autres
Publié: (2024)

Completed Hyperparameter Transfer across Modules, Width, Depth, Batch and Duration
par: Mlodozeniec, Bruno, et autres
Publié: (2025)

Shielded Diffusion: Generating Novel and Diverse Images using Sparse Repellency
par: Kirchhof, Michael, et autres
Publié: (2024)

Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters
par: Guo, Zhiyu, et autres
Publié: (2024)

In-context KV-Cache Eviction for LLMs via Attention-Gate
par: Zeng, Zihao, et autres
Publié: (2024)

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
par: Deng, Difan, et autres
Publié: (2026)

Attention with Trained Embeddings Provably Selects Important Tokens
par: Wu, Diyuan, et autres
Publié: (2025)

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
par: Bai, Yushi, et autres
Publié: (2026)

Sparse Attention across Multiple-context KV Cache
par: Cao, Ziyi, et autres
Publié: (2025)

Cubit: Token Mixer with Kernel Ridge Regression
par: Zheng, Chuanyang, et autres
Publié: (2026)

Token Distillation: Attention-aware Input Embeddings For New Tokens
par: Dobler, Konstantin, et autres
Publié: (2025)

LoLA: Low-Rank Linear Attention With Sparse Caching
par: McDermott, Luke, et autres
Publié: (2025)

AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
par: Yang, Qingyue, et autres
Publié: (2025)

Optimal Splitting of Language Models from Mixtures to Specialized Domains
par: Seto, Skyler, et autres
Publié: (2026)

ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
par: Ye, Lu, et autres
Publié: (2024)

Auto-Regressive Next-Token Predictors are Universal Learners
par: Malach, Eran
Publié: (2023)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
par: Brandon, William, et autres
Publié: (2024)

RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
par: Tang, Hanlin, et autres
Publié: (2024)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
par: Jo, Dongwon, et autres
Publié: (2026)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
par: Wu, Wei, et autres
Publié: (2024)

SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget
par: Wang, Zihao, et autres
Publié: (2024)

Softmax Attention with Constant Cost per Token
par: Heinsen, Franz A.
Publié: (2024)

HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs
par: Yang, Dongquan, et autres
Publié: (2025)

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
par: Saxena, Utkarsh, et autres
Publié: (2024)

Multivariate Conformal Prediction using Optimal Transport
par: Klein, Michal, et autres
Publié: (2025)