Imagen de Portada

Fast Multipole Attention: A Scalable Multilevel Attention Mechanism for Text and Images

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kang, Yanming, Tran, Giang, De Sterck, Hans
Formato:	Preprint
Publicado:	2023
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Multipole Attention for Efficient Long Context Reasoning
por: Hooper, Coleman, et al.
Publicado: (2025)

Hardware-Efficient Attention for Fast Decoding
por: Zadouri, Ted, et al.
Publicado: (2025)

BiSparse-AAS: Bilinear Sparse Attention and Adaptive Spans Framework for Scalable and Efficient Text Summarization
por: Hagos, Desta Haileselassie, et al.
Publicado: (2025)

Scalable-Softmax Is Superior for Attention
por: Nakanishi, Ken M.
Publicado: (2025)

Learning to Focus: Focal Attention for Selective and Scalable Transformers
por: Ram, Dhananjay, et al.
Publicado: (2025)

Sparse Query Attention (SQA): A Computationally Efficient Attention Mechanism with Query Heads Reduction
por: Filipek, Adam
Publicado: (2025)

Generalized Probabilistic Attention Mechanism in Transformers
por: Heo, DongNyeong, et al.
Publicado: (2024)

AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms
por: Chen, Feiyang, et al.
Publicado: (2025)

Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics
por: Colagrande, Alex, et al.
Publicado: (2025)

Limitations of Normalization in Attention Mechanism
por: Mudarisov, Timur, et al.
Publicado: (2025)

Adaptive Two Sided Laplace Transforms: A Learnable, Interpretable, and Scalable Replacement for Self-Attention
por: Kiruluta, Andrew
Publicado: (2025)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
por: Musat, Tiberiu
Publicado: (2024)

Detecting Suicidal Ideation in Text with Interpretable Deep Learning: A CNN-BiGRU with Attention Mechanism
por: Bhuiyan, Mohaiminul Islam, et al.
Publicado: (2025)

Why Softmax Attention Outperforms Linear Attention
por: Deng, Yichuan, et al.
Publicado: (2023)

SEA: Sparse Linear Attention with Estimated Attention Mask
por: Lee, Heejun, et al.
Publicado: (2023)

FLuRKA: Fast and accurate unified Low-Rank & Kernel Attention
por: Gupta, Ahan, et al.
Publicado: (2023)

Paying Attention to Facts: Quantifying the Knowledge Capacity of Attention Layers
por: Wong, Liang Ze
Publicado: (2025)

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
por: Lai, Xunhao, et al.
Publicado: (2025)

Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference
por: Qiu, Quantong, et al.
Publicado: (2026)

Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank
por: Roy, Debjyoti Saha, et al.
Publicado: (2024)

PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention
por: Chen, Lida, et al.
Publicado: (2025)

A Simple Attention-Based Mechanism for Bimodal Emotion Classification
por: Elabd, Mazen, et al.
Publicado: (2024)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
por: Zhang, Stephen, et al.
Publicado: (2025)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
por: Jo, Dongwon, et al.
Publicado: (2026)

Learning to Attribute with Attention
por: Cohen-Wang, Benjamin, et al.
Publicado: (2025)

Exclusive Self Attention
por: Zhai, Shuangfei
Publicado: (2026)

Scale-invariant Attention
por: Anson, Ben, et al.
Publicado: (2025)

Breaking the Attention Bottleneck
por: Hilsenbek, Kalle
Publicado: (2024)

Scaling Bidirectional Spans and Span Violations in Attention Mechanism
por: Kim, Jongwook, et al.
Publicado: (2025)

Stochastic Attention: Connectome-Inspired Randomized Routing for Expressive Linear-Time Attention
por: Jin, Zehao, et al.
Publicado: (2026)

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
por: Shyam, Vasudev, et al.
Publicado: (2024)

Simulating Hard Attention Using Soft Attention
por: Yang, Andy, et al.
Publicado: (2024)

Scalable Efficient Training of Large Language Models with Low-dimensional Projected Attention
por: Lv, Xingtai, et al.
Publicado: (2024)

Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
por: Chen, Yen-Shan, et al.
Publicado: (2025)

S$^3$-Attention:Attention-Aligned Endogenous Retrieval for Memory-Bounded Long-Context Inference
por: Ma, Qingsen, et al.
Publicado: (2026)

AttnCache: Accelerating Self-Attention Inference for LLM Prefill via Attention Cache
por: Song, Dinghong, et al.
Publicado: (2025)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
por: He, Mutian, et al.
Publicado: (2025)

Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
por: Deng, Difan, et al.
Publicado: (2026)

Enhanced Structured State Space Models via Grouped FIR Filtering and Attention Sink Mechanisms
por: Meng, Tian, et al.
Publicado: (2024)

Optimizing Mixture of Block Attention
por: Xiao, Guangxuan, et al.
Publicado: (2025)