Cover Image

The emergence of sparse attention: impact of data distribution and benefits of repetition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zucchet, Nicolas, d'Angelo, Francesco, Lampinen, Andrew K., Chan, Stephanie C. Y.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Neural and Evolutionary Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Gated recurrent neural networks discover attention
by: Zucchet, Nicolas, et al.
Published: (2023)

Joint sparse coding and temporal dynamics support context reconfiguration
by: Shi, Qianqian, et al.
Published: (2026)

Gate-level boolean evolutionary geometric attention neural networks
by: Shi, Xianshuai, et al.
Published: (2025)

SkipSNN: Efficiently Classifying Spike Trains with Event-attention
by: Yin, Hang, et al.
Published: (2024)

A Neural-Evolutionary Algorithm for Autonomous Transit Network Design
by: Holliday, Andrew, et al.
Published: (2024)

Differentiable architecture search with multi-dimensional attention for spiking neural networks
by: Man, Yilei, et al.
Published: (2024)

Incremental Affinity Propagation based on Cluster Consolidation and Stratification
by: Castano, Silvana, et al.
Published: (2024)

Temporal-Difference Learning Using Distributed Error Signals
by: Guan, Jonas, et al.
Published: (2024)

Particle swarm optimization for online sparse streaming feature selection under uncertainty
by: Xu, Ruiyang
Published: (2025)

Deep greedy unfolding: Sorting out argsorting in greedy sparse recovery algorithms
by: Mohammad-Taheri, Sina, et al.
Published: (2025)

Combining Relevance and Magnitude for Resource-Aware DNN Pruning
by: Chiasserini, Carla Fabiana, et al.
Published: (2024)

FAGH: Accelerating Federated Learning with Approximated Global Hessian
by: Sen, Mrinmay, et al.
Published: (2024)

Cultivating Archipelago of Forests: Evolving Robust Decision Trees through Island Coevolution
by: Żychowski, Adam, et al.
Published: (2024)

DALex: Lexicase-like Selection via Diverse Aggregation
by: Ni, Andrew, et al.
Published: (2024)

A Critical Analysis of the Theoretical Framework of the Extreme Learning Machine
by: Perfilievaa, Irina, et al.
Published: (2024)

Genetic Programming for Explainable Manifold Learning
by: Cravens, Ben, et al.
Published: (2024)

Reservoir Computing via Multi-Scale Random Fourier Features for Forecasting Fast-Slow Dynamical Systems
by: Laha, S. K.
Published: (2025)

Multi-objective evolutionary GAN for tabular data synthesis
by: Ran, Nian, et al.
Published: (2024)

Hypercomplex neural network in time series forecasting of stock data
by: Kycia, Radosław, et al.
Published: (2024)

Probabilistic computation and uncertainty quantification with emerging covariance
by: Ma, Hengyuan, et al.
Published: (2023)

Closed-Form Test Functions for Biophysical Sequence Optimization Algorithms
by: Stanton, Samuel, et al.
Published: (2024)

Statistical Properties of the King Wen Sequence: An Anti-Habituation Structure That Does Not Improve Neural Network Training
by: Chan, Augustin
Published: (2026)

SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrix
by: Sen, Mrinmay, et al.
Published: (2024)

Neuromorphic on-chip reservoir computing with spiking neural network architectures
by: Karki, Samip, et al.
Published: (2024)

MC-QDSNN: Quantized Deep evolutionary SNN with Multi-Dendritic Compartment Neurons for Stress Detection using Physiological Signals
by: S, Ajay B, et al.
Published: (2024)

SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks
by: Venkatesh, Sreyes, et al.
Published: (2024)

ToolMol: Evolutionary Agentic Framework for Multi-objective Drug Discovery
by: Zhou, Andrew Y., et al.
Published: (2026)

Dataset-learning duality and emergent criticality
by: Kukleva, Ekaterina, et al.
Published: (2024)

Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learning
by: Velarde, Osvaldo M, et al.
Published: (2024)

A White-Box SVM Framework and its Swarm-Based Optimization for Supervision of Toothed Milling Cutter through Characterization of Spindle Vibrations
by: Deo, Tejas Y., et al.
Published: (2021)

LeanKAN: A Parameter-Lean Kolmogorov-Arnold Network Layer with Improved Memory Efficiency and Convergence Behavior
by: Koenig, Benjamin C., et al.
Published: (2025)

Enhancing Imbalance Learning: A Novel Slack-Factor Fuzzy SVM Approach
by: Tanveer, M., et al.
Published: (2024)

Analysis and Explainability of LLMs Via Evolutionary Methods
by: Gallagher, Shannon K., et al.
Published: (2026)

A Complete Pipeline for deploying SNNs with Synaptic Delays on Loihi 2
by: Mészáros, Balázs, et al.
Published: (2025)

ActNAS : Generating Efficient YOLO Models using Activation NAS
by: Sah, Sudhakar, et al.
Published: (2024)

ReLiCADA -- Reservoir Computing using Linear Cellular Automata Design Algorithm
by: Kantic, Jonas, et al.
Published: (2023)

Discount Model Search for Quality Diversity Optimization in High-Dimensional Measure Spaces
by: Tjanaka, Bryon, et al.
Published: (2026)

Density Descent for Diversity Optimization
by: Lee, David H., et al.
Published: (2023)

CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection
by: Darban, Zahra Zamanzadeh, et al.
Published: (2023)

Supervised Spike Agreement Dependent Plasticity for Fast Local Learning in Spiking Neural Networks
by: S, Gouri Lakshmi, et al.
Published: (2026)