Cover Image

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Jintao, Xiang, Chendong, Huang, Haofeng, Wei, Jia, Xi, Haocheng, Zhu, Jun, Chen, Jianfei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Performance
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning
by: Zhang, Jintao, et al.
Published: (2026)

SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training
by: Zhang, Jintao, et al.
Published: (2025)

SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization
by: Zhang, Jintao, et al.
Published: (2024)

SageAttention2++: A More Efficient Implementation of SageAttention2
by: Zhang, Jintao, et al.
Published: (2025)

SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
by: Zhang, Jintao, et al.
Published: (2024)

Towards Efficient Multi-Scale Deformable Attention on NPU
by: Huang, Chenghuan, et al.
Published: (2025)

Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation
by: Yang, Shuo, et al.
Published: (2025)

Minuet: Accelerating 3D Sparse Convolutions on GPUs
by: Yang, Jiacheng, et al.
Published: (2023)

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
by: Zhang, Jintao, et al.
Published: (2025)

Oscillation-Reduced MXFP4 Training for Vision Transformers
by: Chen, Yuxiang, et al.
Published: (2025)

XAttention: Block Sparse Attention with Antidiagonal Scoring
by: Xu, Ruyi, et al.
Published: (2025)

VecAttention: Vector-wise Sparse Attention for Accelerating Long Context Inference
by: Liu, Anmin, et al.
Published: (2026)

TakuNet: an Energy-Efficient CNN for Real-Time Inference on Embedded UAV systems in Emergency Response Scenarios
by: Rossi, Daniel, et al.
Published: (2025)

SparseInfer: Training-free Prediction of Activation Sparsity for Fast LLM Inference
by: Shin, Jiho, et al.
Published: (2024)

SLA2: Sparse-Linear Attention with Learnable Routing and QAT
by: Zhang, Jintao, et al.
Published: (2026)

TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization
by: Zhong, Chonghao, et al.
Published: (2026)

Performance is not All You Need: Sustainability Considerations for Algorithms
by: Li, Xiang, et al.
Published: (2025)

VSA: Faster Video Diffusion with Trainable Sparse Attention
by: Zhang, Peiyuan, et al.
Published: (2025)

A Study on Inference Latency for Vision Transformers on Mobile Devices
by: Li, Zhuojin, et al.
Published: (2025)

H2EAL: Hybrid-Bonding Architecture with Hybrid Sparse Attention for Efficient Long-Context LLM Inference
by: Fu, Zizhuo, et al.
Published: (2025)

PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference
by: Fang, Jiarui, et al.
Published: (2024)

FreeTuner: Any Subject in Any Style with Training-free Diffusion
by: Xu, Youcan, et al.
Published: (2024)

FlashOmni: A Unified Sparse Attention Engine for Diffusion Transformers
by: Qiao, Liang, et al.
Published: (2025)

Training-free and Adaptive Sparse Attention for Efficient Long Video Generation
by: Xia, Yifei, et al.
Published: (2025)

KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
by: Wang, Han, et al.
Published: (2026)

An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference
by: Yao, Feiyu, et al.
Published: (2026)

AnyMS: Bottom-up Attention Decoupling for Layout-guided and Training-free Multi-subject Customization
by: Yu, Binhe, et al.
Published: (2025)

Block Sparse Flash Attention
by: Ohayon, Daniel, et al.
Published: (2025)

Bidirectional Sparse Attention for Faster Video Diffusion Training
by: Zhan, Chenlu, et al.
Published: (2025)

MAUP: Training-free Multi-center Adaptive Uncertainty-aware Prompting for Cross-domain Few-shot Medical Image Segmentation
by: Zhu, Yazhou, et al.
Published: (2025)

QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference
by: Zhang, Miao, et al.
Published: (2026)

A Methodology to Evaluate Strategies Predicting Rankings on Unseen Domains
by: Piérard, Sébastien, et al.
Published: (2025)

Enhancing Traffic Sign Recognition On The Performance Based On Yolov8
by: Ibrahim, Baba, et al.
Published: (2025)

A Diffusion-Based Framework for Configurable and Realistic Multi-Storage Trace Generation
by: Kim, Seohyun, et al.
Published: (2025)

Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices
by: Ghafouri, Saeid, et al.
Published: (2025)

Resource-Efficient RGB-Only Action Recognition for Edge Deployment
by: Yoon, Dongsik, et al.
Published: (2026)

Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models
by: Cheng, Hao, et al.
Published: (2024)

CPU Optimization of a Monocular 3D Biomechanics Pipeline for Low-Resource Deployment
by: Zhang, Yan, et al.
Published: (2026)

ConvBench: A Comprehensive Benchmark for 2D Convolution Primitive Evaluation
by: Alvarenga, Lucas, et al.
Published: (2024)

DynamicRad: Content-Adaptive Sparse Attention for Long Video Diffusion
by: Long, Yongji, et al.
Published: (2026)