Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | |
| Published: |
Zenodo
2025
|
| Online Access: | https://doi.org/10.5281/zenodo.17137381 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866901123553558528 |
|---|---|
| author | Willerton, Freeman P. |
| author_facet | Willerton, Freeman P. |
| contents | <p>This deposit accompanies two complementary documents that together formalize and operationalize a new framework for attention mechanisms. The first document develops the theoretical foundation, introducing operator-theoretic, multi-scale, and neural-symbolic generalizations of self-attention. Key contributions include renormalisation-group attention flows, adaptive interpolation between vector-based and symbolic affinities, product-of-experts gating, entropy-based token budgeting, and prime-indexed positional encodings. The second document provides an operational addendum, translating these theoretical advances into precise, testable specifications suitable for deterministic software implementations. It details invariants, edge-case handling, and verification protocols, ensuring robustness, safety, and reproducibility. Together, the works extend the classical affinity-matrix paradigm, offering a mathematically principled and scalable approach to efficient and expressive neural attention.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17137381 |
| institution | Zenodo |
| language | |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Advances in Affinity-Based Attention (Program Snapshot - Sept 2025) Willerton, Freeman P. <p>This deposit accompanies two complementary documents that together formalize and operationalize a new framework for attention mechanisms. The first document develops the theoretical foundation, introducing operator-theoretic, multi-scale, and neural-symbolic generalizations of self-attention. Key contributions include renormalisation-group attention flows, adaptive interpolation between vector-based and symbolic affinities, product-of-experts gating, entropy-based token budgeting, and prime-indexed positional encodings. The second document provides an operational addendum, translating these theoretical advances into precise, testable specifications suitable for deterministic software implementations. It details invariants, edge-case handling, and verification protocols, ensuring robustness, safety, and reproducibility. Together, the works extend the classical affinity-matrix paradigm, offering a mathematically principled and scalable approach to efficient and expressive neural attention.</p> |
| title | Advances in Affinity-Based Attention (Program Snapshot - Sept 2025) |
| url | https://doi.org/10.5281/zenodo.17137381 |