Cover Image

A Mathematical Theory of Top-$k$ Sparse Attention via Total Variation Distance

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tzachristas, Georgios, Deng, Lei, Tzachristas, Ioannis, Zhang, Gong, Chen, Renhai
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

NOEM$^{3}$A: A Neuro-Symbolic Ontology-Enhanced Method for Multi-Intent Understanding in Mobile Agents
by: Tzachristas, Ioannis, et al.
Published: (2025)

Guided Persona-based AI Surveys: Can we replicate personal mobility preferences at scale using LLMs?
by: Tzachristas, Ioannis, et al.
Published: (2025)

Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
by: Tzachristas, Ioannis
Published: (2024)

HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
by: Gong, Ping, et al.
Published: (2025)

ZETA: Leveraging Z-order Curves for Efficient Top-k Attention
by: Zeng, Qiuhao, et al.
Published: (2025)

Towards Robust Knowledge Tracing Models via k-Sparse Attention
by: Huang, Shuyan, et al.
Published: (2024)

Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs
by: Ni, Wentao, et al.
Published: (2026)

Optimizing Partial Area Under the Top-k Curve: Theory and Practice
by: Wang, Zitai, et al.
Published: (2022)

Intrinsically Interpretable Attention via Sparse Post-Training
by: Draye, Florent, et al.
Published: (2025)

BatchTopK Sparse Autoencoders
by: Bussmann, Bart, et al.
Published: (2024)

How Sparse Attention Approximates Exact Attention? Your Attention is Naturally $n^C$-Sparse
by: Deng, Yichuan, et al.
Published: (2024)

S2O: Early Stopping for Sparse Attention via Online Permutation
by: Zhang, Yu, et al.
Published: (2026)

ATP: Enabling Fast LLM Serving via Attention on Top Principal Keys
by: Niu, Yue, et al.
Published: (2024)

EMA Policy Gradient: Taming Reinforcement Learning for LLMs with EMA Anchor and Top-k KL
by: Zhang, Lunjun, et al.
Published: (2026)

vAttention: Verified Sparse Attention
by: Desai, Aditya, et al.
Published: (2025)

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
by: Gao, Yizhao, et al.
Published: (2025)

Improving Sparse Autoencoder with Dynamic Attention
by: Wang, Dongsheng, et al.
Published: (2026)

Sparse Variational Student-t Processes
by: Xu, Jian, et al.
Published: (2023)

Foundations of Top-$k$ Decoding For Language Models
by: Noarov, Georgy, et al.
Published: (2025)

Personalized Top-k Set Queries Over Predicted Scores
by: Nia, Sohrab Namazi, et al.
Published: (2025)

Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys
by: Yang, Xu, et al.
Published: (2026)

HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference
by: L, Yashas Samaga B, et al.
Published: (2024)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
by: Yuan, Jingyang, et al.
Published: (2025)

DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention
by: Huang, Yuxiang, et al.
Published: (2026)

VSPrefill: Vertical-Slash Sparse Attention with Lightweight Indexing for Long-Context Prefilling
by: Guanzhong, Chen
Published: (2026)

Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes
by: Chen, Yingyi, et al.
Published: (2024)

Enhancing Performance for Highly Imbalanced Medical Data via Data Regularization in a Federated Learning Setting
by: Tsoumplekas, Georgios, et al.
Published: (2024)

$\mathbb{R}^{2k}$ is Theoretically Large Enough for Embedding-based Top-$k$ Retrieval
by: Wang, Zihao, et al.
Published: (2026)

SparseBalance: Load-Balanced Long Context Training with Dynamic Sparse Attention
by: Xu, Hongtao, et al.
Published: (2026)

From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
by: Ren, Yuxin, et al.
Published: (2026)

AbsTopK: Rethinking Sparse Autoencoders For Bidirectional Features
by: Zhu, Xudong, et al.
Published: (2025)

Explainable fault and severity classification for rolling element bearings using Kolmogorov-Arnold networks
by: Rigas, Spyros, et al.
Published: (2024)

Sparse Variational Student-t Processes for Heavy-tailed Modeling
by: Xu, Jian, et al.
Published: (2024)

TANGNN: a Concise, Scalable and Effective Graph Neural Networks with Top-m Attention Mechanism for Graph Representation Learning
by: E, Jiawei, et al.
Published: (2024)

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
by: Xu, Yufei, et al.
Published: (2026)

HSR-Enhanced Sparse Attention Acceleration
by: Chen, Bo, et al.
Published: (2024)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
by: Zeng, Zhiyuan, et al.
Published: (2024)

MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference
by: Zhou, Ruijie, et al.
Published: (2026)

Antithetic Sampling for Top-k Shapley Identification
by: Kolpaczki, Patrick, et al.
Published: (2025)

Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
by: Zhou, Qihui, et al.
Published: (2025)