Image de couverture de livre

DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Jin, Haolin, Xiao, Mengbai, Yuan, Yuan, Zhang, Xiao, Yu, Dongxiao, Zhang, Guanghui, Wang, Haoliang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding
par: Yin, Haofei, et autres
Publié: (2025)

TAMO: Fine-Grained Root Cause Analysis via Tool-Assisted LLM Agent with Multi-Modality Observation Data in Cloud-Native Systems
par: Zhang, Xiao, et autres
Publié: (2025)

COMCAT: Towards Efficient Compression and Customization of Attention-Based Vision Models
par: Xiao, Jinqi, et autres
Publié: (2023)

Revealing the Attention Floating Mechanism in Masked Diffusion Models
par: Dai, Xin, et autres
Publié: (2026)

Data-Free Continual Learning of Server Models in Model-Heterogeneous Cloud-Device Collaboration
par: Zhang, Xiao, et autres
Publié: (2025)

EffiCANet: Efficient Time Series Forecasting with Convolutional Attention
par: Zhou, Xinxing, et autres
Publié: (2024)

Fairness-aware Vision Transformer via Debiased Self-Attention
par: Qiang, Yao, et autres
Publié: (2023)

D-FaST: Cognitive Signal Decoding with Disentangled Frequency-Spatial-Temporal Attention
par: Chen, Weiguo, et autres
Publié: (2024)

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
par: Yuan, Jingyang, et autres
Publié: (2025)

PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting
par: Zhao, Bowen, et autres
Publié: (2025)

Attention Smoothing Is All You Need For Unlearning
par: Zade, Saleh Zare, et autres
Publié: (2026)

Quantum Phase Recognition via Quantum Attention Mechanism
par: Chen, Jin-Long, et autres
Publié: (2026)

Self-Attention Mechanism in Multimodal Context for Banking Transaction Flow
par: Delestre, Cyrile, et autres
Publié: (2024)

A Neural Network Architecture Based on Attention Gate Mechanism for 3D Magnetotelluric Forward Modeling
par: Zhong, Xin, et autres
Publié: (2025)

MoH: Multi-Head Attention as Mixture-of-Head Attention
par: Jin, Peng, et autres
Publié: (2024)

HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference
par: Gong, Ping, et autres
Publié: (2025)

CSAI: Conditional Self-Attention Imputation for Healthcare Time-series
par: Qian, Linglong, et autres
Publié: (2023)

Attention Sinks and Outliers in Attention Residuals
par: Luo, Haozheng, et autres
Publié: (2026)

FAMOUS: Flexible Accelerator for the Attention Mechanism of Transformer on UltraScale+ FPGAs
par: Kabir, Ehsan, et autres
Publié: (2024)

Continuous-Time Attention: PDE-Guided Mechanisms for Long-Sequence Transformers
par: Zhang, Yukun, et autres
Publié: (2025)

Spatial-Temporal Attention Model for Traffic State Estimation with Sparse Internet of Vehicles
par: Xue, Jianzhe, et autres
Publié: (2024)

Feature Fusion Based on Mutual-Cross-Attention Mechanism for EEG Emotion Recognition
par: Zhao, Yimin, et autres
Publié: (2024)

Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems
par: Amizadeh, Saeed, et autres
Publié: (2025)

Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation
par: Racioppo, Peter
Publié: (2025)

Native Hybrid Attention for Efficient Sequence Modeling
par: Du, Jusen, et autres
Publié: (2025)

Tensor Product Attention Is All You Need
par: Zhang, Yifan, et autres
Publié: (2025)

PSformer: Parameter-efficient Transformer with Segment Attention for Time Series Forecasting
par: Wang, Yanlong, et autres
Publié: (2024)

ATTENTION2D: Communication Efficient Distributed Self-Attention Mechanism
par: Elango, Venmugil
Publié: (2025)

Not All Attention Heads Are What You Need: Refining CLIP's Image Representation with Attention Ablation
par: Lin, Feng, et autres
Publié: (2025)

Efficient Linear Attention for Multivariate Time Series Modeling via Entropy Equality
par: Zhang, Mingtao, et autres
Publié: (2025)

Modeling Choice via Self-Attention
par: Ko, Joohwan, et autres
Publié: (2023)

Quaternion Self-Attention with Shared Scores
par: Yamauchi, Shogo, et autres
Publié: (2026)

Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfaces
par: Wang, Jiyuan, et autres
Publié: (2025)

Self-Indexing KVCache: Predicting Sparse Attention from Compressed Keys
par: Yang, Xu, et autres
Publié: (2026)

AttentionDep: Domain-Aware Attention for Explainable Depression Severity Assessment
par: Ibrahimov, Yusif, et autres
Publié: (2025)

HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention
par: Xu, Yufei, et autres
Publié: (2026)

Toward Efficient Permutation for Hierarchical N:M Sparsity on GPUs
par: Yu, Seungmin, et autres
Publié: (2024)

FADTI: Fourier and Attention Driven Diffusion for Multivariate Time Series Imputation
par: Li, Runze, et autres
Publié: (2025)

HGATSolver: A Heterogeneous Graph Attention Solver for Fluid-Structure Interaction
par: Zhang, Qin-Yi, et autres
Publié: (2026)

MoEBlaze: Breaking the Memory Wall for Efficient MoE Training on Modern GPUs
par: Zhang, Jiyuan, et autres
Publié: (2026)