A Novel Spinor-Based Embedding Model for Transformers
Fuente:
arXiv
Enregistré dans:
| Auteur principal: | |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912052641005568 |
|---|---|
| author | White, Rick |
| author_facet | White, Rick |
| contents | This paper proposes a novel approach to word embeddings in Transformer models by utilizing spinors from geometric algebra. Spinors offer a rich mathematical framework capable of capturing complex relationships and transformations in high-dimensional spaces. By encoding words as spinors, we aim to enhance the expressiveness and robustness of language representations. We present the theoretical foundations of spinors, detail their integration into Transformer architectures, and discuss potential advantages and challenges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_00038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Novel Spinor-Based Embedding Model for Transformers White, Rick Machine Learning Computation and Language This paper proposes a novel approach to word embeddings in Transformer models by utilizing spinors from geometric algebra. Spinors offer a rich mathematical framework capable of capturing complex relationships and transformations in high-dimensional spaces. By encoding words as spinors, we aim to enhance the expressiveness and robustness of language representations. We present the theoretical foundations of spinors, detail their integration into Transformer architectures, and discuss potential advantages and challenges. |
| title | A Novel Spinor-Based Embedding Model for Transformers |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2410.00038 |