Resonant Sparse Geometry Networks
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915755433394176 |
|---|---|
| author | Hays, Hasi |
| author_facet | Hays, Hasi |
| contents | We introduce Resonant Sparse Geometry Networks (RSGN), a brain-inspired architecture with self-organizing sparse
hierarchical input-dependent connectivity. Unlike Transformer architectures that employ dense attention mechanisms with
O(n^2) computational complexity, RSGN embeds computational nodes in learned hyperbolic space where connection strength
decays with geodesic distance, achieving dynamic sparsity that adapts to each input. The architecture operates on two
distinct timescales: fast differentiable activation propagation optimized through gradient descent, and slow
Hebbian-inspired structural learning for connectivity adaptation through local correlation rules. We provide rigorous
mathematical analysis demonstrating that RSGN achieves O(n*k) computational complexity, where k << n represents the average
active neighborhood size. Experimental evaluation on hierarchical classification and long-range dependency tasks
demonstrates that RSGN achieves 96.5% accuracy on long-range dependency tasks while using approximately 15x fewer
parameters than standard Transformers. On challenging hierarchical classification with 20 classes, RSGN achieves 23.8%
accuracy (compared to 5% random baseline) with only 41,672 parameters, nearly 10x fewer than the Transformer baselines
which require 403,348 parameters to achieve 30.1% accuracy. Our ablation studies confirm the contribution of each architectural
component, with Hebbian learning providing consistent improvements. These results suggest that brain-inspired principles
of sparse, geometrically-organized computation offer a promising direction toward more efficient and biologically plausible
neural architectures. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18064 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Resonant Sparse Geometry Networks Hays, Hasi Machine Learning Artificial Intelligence Neural and Evolutionary Computing We introduce Resonant Sparse Geometry Networks (RSGN), a brain-inspired architecture with self-organizing sparse hierarchical input-dependent connectivity. Unlike Transformer architectures that employ dense attention mechanisms with O(n^2) computational complexity, RSGN embeds computational nodes in learned hyperbolic space where connection strength decays with geodesic distance, achieving dynamic sparsity that adapts to each input. The architecture operates on two distinct timescales: fast differentiable activation propagation optimized through gradient descent, and slow Hebbian-inspired structural learning for connectivity adaptation through local correlation rules. We provide rigorous mathematical analysis demonstrating that RSGN achieves O(n*k) computational complexity, where k << n represents the average active neighborhood size. Experimental evaluation on hierarchical classification and long-range dependency tasks demonstrates that RSGN achieves 96.5% accuracy on long-range dependency tasks while using approximately 15x fewer parameters than standard Transformers. On challenging hierarchical classification with 20 classes, RSGN achieves 23.8% accuracy (compared to 5% random baseline) with only 41,672 parameters, nearly 10x fewer than the Transformer baselines which require 403,348 parameters to achieve 30.1% accuracy. Our ablation studies confirm the contribution of each architectural component, with Hebbian learning providing consistent improvements. These results suggest that brain-inspired principles of sparse, geometrically-organized computation offer a promising direction toward more efficient and biologically plausible neural architectures. |
| title | Resonant Sparse Geometry Networks |
| topic | Machine Learning Artificial Intelligence Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2601.18064 |