Transforming Hidden States into Binary Semantic Features
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913522751897600 |
|---|---|
| author | Musil, Tomáš Mareček, David |
| author_facet | Musil, Tomáš Mareček, David |
| contents | Large language models follow a lineage of many NLP applications that were directly inspired by distributional semantics, but do not seem to be closely related to it anymore. In this paper, we propose to employ the distributional theory of meaning once again. Using Independent Component Analysis to overcome some of its challenging aspects, we show that large language models represent semantic features in their hidden states. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19813 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Transforming Hidden States into Binary Semantic Features Musil, Tomáš Mareček, David Computation and Language Large language models follow a lineage of many NLP applications that were directly inspired by distributional semantics, but do not seem to be closely related to it anymore. In this paper, we propose to employ the distributional theory of meaning once again. Using Independent Component Analysis to overcome some of its challenging aspects, we show that large language models represent semantic features in their hidden states. |
| title | Transforming Hidden States into Binary Semantic Features |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.19813 |