Memory Mosaics
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916634456752128 |
|---|---|
| author | Zhang, Jianyu Nolte, Niklas Sadhukhan, Ranajoy Chen, Beidi Bottou, Léon |
| author_facet | Zhang, Jianyu Nolte, Niklas Sadhukhan, Ranajoy Chen, Beidi Bottou, Léon |
| contents | Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilities and in-context learning capabilities. Unlike transformers, memory mosaics achieve these capabilities in comparatively transparent way ("predictive disentanglement"). We illustrate these capabilities on a toy example and also show that memory mosaics perform as well or better than transformers on medium-scale language modeling tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_06394 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Memory Mosaics Zhang, Jianyu Nolte, Niklas Sadhukhan, Ranajoy Chen, Beidi Bottou, Léon Machine Learning Artificial Intelligence Neural and Evolutionary Computing Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilities and in-context learning capabilities. Unlike transformers, memory mosaics achieve these capabilities in comparatively transparent way ("predictive disentanglement"). We illustrate these capabilities on a toy example and also show that memory mosaics perform as well or better than transformers on medium-scale language modeling tasks. |
| title | Memory Mosaics |
| topic | Machine Learning Artificial Intelligence Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2405.06394 |