Linear Attention as Bayesian Inference
Fuente:
Zenodo
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901742876098560 |
|---|---|
| author | Frimane, Âzeddine |
| author_facet | Frimane, Âzeddine |
| contents | <p>This paper shows that the recurrent update used in linear attention models is mathematically equivalent to Bayesian inference under a Dependent Dirichlet Process. The forgetting rate, the input-dependent gate, and the multi-head structure each receive a clean probabilistic interpretation. The result is exact and holds by construction, not as an approximation.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18975350 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Linear Attention as Bayesian Inference Frimane, Âzeddine linear attention, Dirichlet process, Bayesian nonparametrics, recurrent neural networks, state space models, exponential smoothing, hierarchical Dirichlet process, sequence modeling <p>This paper shows that the recurrent update used in linear attention models is mathematically equivalent to Bayesian inference under a Dependent Dirichlet Process. The forgetting rate, the input-dependent gate, and the multi-head structure each receive a clean probabilistic interpretation. The result is exact and holds by construction, not as an approximation.</p> |
| title | Linear Attention as Bayesian Inference |
| topic | linear attention, Dirichlet process, Bayesian nonparametrics, recurrent neural networks, state space models, exponential smoothing, hierarchical Dirichlet process, sequence modeling |
| url | https://doi.org/10.5281/zenodo.18975350 |