Cover Image

Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hassani, Ali, Hwu, Wen-Mei, Shi, Humphrey
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Generalized Neighborhood Attention: Multi-dimensional Sparse Attention at the Speed of Light
by: Hassani, Ali, et al.
Published: (2025)

Efficient Image Generation with Variadic Attention Heads
by: Walton, Steven, et al.
Published: (2022)

FasterViT: Fast Vision Transformers with Hierarchical Attention
by: Hatamizadeh, Ali, et al.
Published: (2023)

Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation
by: Li, Xingyang, et al.
Published: (2025)

From $\mathcal{O}(n^{2})$ to $\mathcal{O}(n)$ Parameters: Quantum Self-Attention in Vision Transformers for Biomedical Image Classification
by: Boucher, Thomas, et al.
Published: (2025)

Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention
by: Hong, Susung
Published: (2024)

Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance
by: Ahn, Donghoon, et al.
Published: (2024)

Human Activity Recognition from Wearable Sensor Data Using Self-Attention
by: Mahmud, Saif, et al.
Published: (2020)

EDNet: Edge-Optimized Small Target Detection in UAV Imagery -- Faster Context Attention, Better Feature Fusion, and Hardware Acceleration
by: Song, Zhifan, et al.
Published: (2025)

Fairness-aware Vision Transformer via Debiased Self-Attention
by: Qiang, Yao, et al.
Published: (2023)

MoH: Multi-Head Attention as Mixture-of-Head Attention
by: Jin, Peng, et al.
Published: (2024)

Attention in Diffusion Model: A Survey
by: Hua, Litao, et al.
Published: (2025)

Self-Attention through Kernel-Eigen Pair Sparse Variational Gaussian Processes
by: Chen, Yingyi, et al.
Published: (2024)

Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Prompt
by: Liu, Chenxi, et al.
Published: (2024)

Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing
by: Mo, Wenyi, et al.
Published: (2024)

PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor
by: Goel, Vidit, et al.
Published: (2023)

Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics
by: Colagrande, Alex, et al.
Published: (2025)

Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation
by: Xu, Jie, et al.
Published: (2025)

Not All Attention Heads Are What You Need: Refining CLIP's Image Representation with Attention Ablation
by: Lin, Feng, et al.
Published: (2025)

MapReduce LoRA: Advancing the Pareto Front in Multi-Preference Optimization for Generative Models
by: Chen, Chieh-Yun, et al.
Published: (2025)

SLA2: Sparse-Linear Attention with Learnable Routing and QAT
by: Zhang, Jintao, et al.
Published: (2026)

CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation
by: Mei, Kangfu, et al.
Published: (2023)

Deep Attention-guided Adaptive Subsampling
by: Shankaranarayana, Sharath M, et al.
Published: (2025)

ENA: Efficient N-dimensional Attention
by: Zhong, Yibo
Published: (2025)

MAIS: Memory-Attention for Interactive Segmentation
by: Orbes-Arteaga, Mauricio, et al.
Published: (2025)

SageAttention2++: A More Efficient Implementation of SageAttention2
by: Zhang, Jintao, et al.
Published: (2025)

TextPixs: Glyph-Conditioned Diffusion with Character-Aware Attention and OCR-Guided Supervision
by: Gillani, Syeda Anshrah, et al.
Published: (2025)

RL for Consistency Models: Faster Reward Guided Text-to-Image Generation
by: Oertell, Owen, et al.
Published: (2024)

Shape-Guided Diffusion with Inside-Outside Attention
by: Park, Dong Huk, et al.
Published: (2022)

Motion meets Attention: Video Motion Prompts
by: Chen, Qixiang, et al.
Published: (2024)

Class-Discriminative Attention Maps for Vision Transformers
by: Brocki, Lennart, et al.
Published: (2023)

DiffCLIP: Differential Attention Meets CLIP
by: Hammoud, Hasan Abed Al Kader, et al.
Published: (2025)

Scratching Visual Transformer's Back with Uniform Attention
by: Hyeon-Woo, Nam, et al.
Published: (2022)

SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference
by: Zhang, Jintao, et al.
Published: (2025)

Dynamic Attention-Guided Diffusion for Image Super-Resolution
by: Moser, Brian B., et al.
Published: (2023)

BSA: Ball Sparse Attention for Large-scale Geometries
by: Brita, Catalin E., et al.
Published: (2025)

Precipitation Nowcasting Using Diffusion Transformer with Causal Attention
by: Li, ChaoRong, et al.
Published: (2024)

Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval
by: Xiao, Ling, et al.
Published: (2022)

Saccade Attention Networks: Using Transfer Learning of Attention to Reduce Network Sizes
by: Estafanous, Marc
Published: (2026)

Regularizing Attention Scores with Bootstrapping
by: Chung, Neo Christopher, et al.
Published: (2026)