Image de couverture de livre

Attention in Constant Time: Vashista Sparse Attention for Long-Context Decoding with Exponential Guarantees

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Nobaub, Vashista
Format:	Preprint
Publié:	2026
Sujets:	Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

GO-OSC and VASH: Geometry-Aware Representation Learning for Early Degradation Detection in Oscillatory Systems
par: Nobaub, Vashista
Publié: (2026)

SparseBalance: Load-Balanced Long Context Training with Dynamic Sparse Attention
par: Xu, Hongtao, et autres
Publié: (2026)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
par: Zhou, Ruijie, et autres
Publié: (2026)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
par: Gao, Yizhao, et autres
Publié: (2025)

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
par: Ni, Wentao, et autres
Publié: (2026)

VSPrefill: Vertical-Slash Sparse Attention with Lightweight Indexing for Long-Context Prefilling
par: Guanzhong, Chen
Publié: (2026)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
par: Yao, Feiyu, et autres
Publié: (2026)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
par: Zhu, Qianchao, et autres
Publié: (2024)

vAttention: Verified Sparse Attention
par: Desai, Aditya, et autres
Publié: (2025)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs
par: Zhu, Kan, et autres
Publié: (2025)

Paged Attention Meets FlexAttention: Unlocking Long-Context Efficiency in Deployed Inference
par: Joshi, Thomas, et autres
Publié: (2025)

MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modeling
par: MiniCPM Team, et autres
Publié: (2026)

TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
par: Yang, Lijie, et autres
Publié: (2024)

Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning
par: Bouadi, Mohamed, et autres
Publié: (2025)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
par: Hu, Junhao, et autres
Publié: (2026)

Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
par: Bu, Tao, et autres
Publié: (2025)

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning
par: Liu, Dong, et autres
Publié: (2026)

RACE Attention: A Strictly Linear-Time Attention Layer for Training on Outrageously Large Contexts
par: Joshi, Sahil, et autres
Publié: (2025)

Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference
par: Deshmukh, Dhruv, et autres
Publié: (2025)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
par: Deng, Yichuan, et autres
Publié: (2024)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
par: Yuan, Jingyang, et autres
Publié: (2025)

Improving Sparse Autoencoder with Dynamic Attention
par: Wang, Dongsheng, et autres
Publié: (2026)

SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
par: Ji, Xiaodong, et autres
Publié: (2025)

SEAL: Scaling to Emphasize Attention for Long-Context Retrieval
par: Lee, Changhun, et autres
Publié: (2025)

Efficient Low Rank Attention for Long-Context Inference in Large Language Models
par: Li, Tenghui, et autres
Publié: (2025)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
par: Huang, Yuxiang, et autres
Publié: (2026)

Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
par: Zhou, Qihui, et autres
Publié: (2025)

HiCI: Hierarchical Construction-Integration for Long-Context Attention
par: Zeng, Xiangyu, et autres
Publié: (2026)

Cost-Optimal Grouped-Query Attention for Long-Context Modeling
par: Chen, Yingfa, et autres
Publié: (2025)

MoBA: Mixture of Block Attention for Long-Context LLMs
par: Lu, Enzhe, et autres
Publié: (2025)

Scaling Context Requires Rethinking Attention
par: Gelada, Carles, et autres
Publié: (2025)

Continuous-Time Attention: PDE-Guided Mechanisms for Long-Sequence Transformers
par: Zhang, Yukun, et autres
Publié: (2025)

Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting
par: Kang, Bong Gyun, et autres
Publié: (2024)

Mixture of Attentions For Speculative Decoding
par: Zimmer, Matthieu, et autres
Publié: (2024)

Stem: Rethinking Causal Information Flow in Sparse Attention
par: Niu, Lin, et autres
Publié: (2026)

Attention-Based Reward Shaping for Sparse and Delayed Rewards
par: Holmes, Ian, et autres
Publié: (2025)

MultiMax: Sparse and Multi-Modal Attention Learning
par: Zhou, Yuxuan, et autres
Publié: (2024)

Intrinsically Interpretable Attention via Sparse Post-Training
par: Draye, Florent, et autres
Publié: (2025)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
par: Ling Team, et autres
Publié: (2025)

XicorAttention: Time Series Transformer Using Attention with Nonlinear Correlation
par: Kimura, Daichi, et autres
Publié: (2025)