Image de couverture de livre

Attentions Under the Microscope: A Comparative Study of Resource Utilization for Variants of Self-Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Tian, Zhengyu, Kumar, Anantha Padmanaban Krishna, Krishnakumar, Hemant, Rawassizadeh, Reza
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Neural and Evolutionary Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems
par: Amizadeh, Saeed, et autres
Publié: (2025)

BiSpikCLM: A Spiking Language Model integrating Softmax-Free Spiking Attention and Spike-Aware Alignment Distillation
par: Guo, Sihang, et autres
Publié: (2026)

Greener GRASS: Enhancing GNNs with Encoding, Rewiring, and Attention
par: Liao, Tongzhou, et autres
Publié: (2024)

Beyond Attention: Toward Machines with Intrinsic Higher Mental States
par: Adeel, Ahsan
Publié: (2025)

Enhancing Graph Representation Learning with Attention-Driven Spiking Neural Networks
par: Yin, Huifeng, et autres
Publié: (2024)

Sparsity Moves Computation: How FFN Architecture Reshapes Attention in Small Transformers
par: Smithline, Gabriel, et autres
Publié: (2026)

SemlaFlow -- Efficient 3D Molecular Generation with Latent Attention and Equivariant Flow Matching
par: Irwin, Ross, et autres
Publié: (2024)

Selective Synchronization Attention
par: Hays, Hasi
Publié: (2026)

Preisach Attention: A Hysteretic Model of Sequential Memory
par: Frydrych, Piotr
Publié: (2026)

GLU Attention Improve Transformer
par: Wang, Zehao
Publié: (2025)

Graph Attention Networks for Detecting Epilepsy from EEG Signals Using Accessible Hardware in Low-Resource Settings
par: Mazurek, Szymon, et autres
Publié: (2025)

Balancing Interpretability and Performance in Motor Imagery EEG Classification: A Comparative Study of ANFIS-FBCSP-PSO and EEGNet
par: Aktar, Farjana, et autres
Publié: (2025)

The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance
par: Rezaei, Mohammad Reza, et autres
Publié: (2025)

Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI
par: Pourcel, Julien, et autres
Publié: (2025)

An Efficient Reconstructed Differential Evolution Variant by Some of the Current State-of-the-art Strategies for Solving Single Objective Bound Constrained Problems
par: Tao, Sichen, et autres
Publié: (2024)

Breaking Global Self-Attention Bottlenecks in Transformer-based Spiking Neural Networks with Local Structure-Aware Self-Attention
par: Li, Lingdong, et autres
Publié: (2026)

A Resource Model For Neural Scaling Law
par: Song, Jinyeop, et autres
Publié: (2024)

Two new feature selection methods based on learn-heuristic techniques for breast cancer prediction: A comprehensive analysis
par: Karimi, Kamyab, et autres
Publié: (2024)

On the Surprising Effectiveness of Attention Transfer for Vision Transformers
par: Li, Alexander C., et autres
Publié: (2024)

Neural Dynamics Self-Attention for Spiking Transformers
par: Zhang, Dehao, et autres
Publié: (2026)

On Accelerating Edge AI: Optimizing Resource-Constrained Environments
par: Sander, Jacob, et autres
Publié: (2025)

NORACL: Neurogenesis for Oracle-free Resource-Adaptive Continual Learning
par: Raghunathan, Karthik Charan, et autres
Publié: (2026)

Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithm
par: Ghader, Mohammadnavid, et autres
Publié: (2025)

CR-BLEA: Contrastive Ranking for Adaptive Resource Allocation in Bilevel Evolutionary Algorithms
par: Xu, Dejun, et autres
Publié: (2025)

An Amortized Efficiency Threshold for Comparing Neural and Heuristic Solvers in Combinatorial Optimization
par: Afifi, Sohaib
Publié: (2026)

Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
par: Chen, Yuzhu, et autres
Publié: (2025)

Stochastic Spiking Attention: Accelerating Attention with Stochastic Computing in Spiking Networks
par: Song, Zihang, et autres
Publié: (2024)

Comparative study of clustering models for multivariate time series from connected medical devices
par: Courrier, Violaine, et autres
Publié: (2023)

Uncertainty Aware Neural Network from Similarity and Sensitivity
par: Kabir, H M Dipu, et autres
Publié: (2023)

Ranking-Enhanced Anomaly Detection Using Active Learning-Assisted Attention Adversarial Dual AutoEncoders
par: Benabderrahmane, Sidahmed, et autres
Publié: (2025)

Towards evolution of Deep Neural Networks through contrastive Self-Supervised learning
par: Vinhas, Adriano, et autres
Publié: (2024)

Large Language Models As Evolution Strategies
par: Lange, Robert Tjarko, et autres
Publié: (2024)

Deep Reinforcement Learning with Spiking Q-learning
par: Chen, Ding, et autres
Publié: (2022)

Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
par: Chen, Ding, et autres
Publié: (2024)

Projective Kolmogorov Arnold Neural Networks (P-KANs): Entropy-Driven Functional Space Discovery for Interpretable Machine Learning
par: Poole, Alastair, et autres
Publié: (2025)

Application of Unsupervised Artificial Neural Network (ANN) Self_Organizing Map (SOM) in Identifying Main Car Sales Factors
par: Taghavi, Mazyar
Publié: (2024)

The Ungrounded Alignment Problem
par: Pickett, Marc, et autres
Publié: (2024)

Position: Leverage Foundational Models for Black-Box Optimization
par: Song, Xingyou, et autres
Publié: (2024)

State-space models can learn in-context by gradient descent
par: Sushma, Neeraj Mohan, et autres
Publié: (2024)

An Experimental Study on Decomposition-Based Deep Ensemble Learning for Traffic Flow Forecasting
par: Zhu, Qiyuan, et autres
Publié: (2024)