Advances in Affinity-Based Attention (Program Snapshot - Sept 2025)
Fuente:
Zenodo
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Recurso digital |
| Publicado: |
Zenodo
2025
|
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866901123553558528 |
|---|---|
| author | Willerton, Freeman P. |
| author_facet | Willerton, Freeman P. |
| contents | <p>This deposit accompanies two complementary documents that together formalize and operationalize a new framework for attention mechanisms. The first document develops the theoretical foundation, introducing operator-theoretic, multi-scale, and neural-symbolic generalizations of self-attention. Key contributions include renormalisation-group attention flows, adaptive interpolation between vector-based and symbolic affinities, product-of-experts gating, entropy-based token budgeting, and prime-indexed positional encodings. The second document provides an operational addendum, translating these theoretical advances into precise, testable specifications suitable for deterministic software implementations. It details invariants, edge-case handling, and verification protocols, ensuring robustness, safety, and reproducibility. Together, the works extend the classical affinity-matrix paradigm, offering a mathematically principled and scalable approach to efficient and expressive neural attention.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17137381 |
| institution | Zenodo |
| language | |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Advances in Affinity-Based Attention (Program Snapshot - Sept 2025) Willerton, Freeman P. <p>This deposit accompanies two complementary documents that together formalize and operationalize a new framework for attention mechanisms. The first document develops the theoretical foundation, introducing operator-theoretic, multi-scale, and neural-symbolic generalizations of self-attention. Key contributions include renormalisation-group attention flows, adaptive interpolation between vector-based and symbolic affinities, product-of-experts gating, entropy-based token budgeting, and prime-indexed positional encodings. The second document provides an operational addendum, translating these theoretical advances into precise, testable specifications suitable for deterministic software implementations. It details invariants, edge-case handling, and verification protocols, ensuring robustness, safety, and reproducibility. Together, the works extend the classical affinity-matrix paradigm, offering a mathematically principled and scalable approach to efficient and expressive neural attention.</p> |
| title | Advances in Affinity-Based Attention (Program Snapshot - Sept 2025) |
| url | https://doi.org/10.5281/zenodo.17137381 |