Are there identifiable structural parts in the sentence embedding whole?
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913413284757504 |
|---|---|
| author | Nastase, Vivi Merlo, Paola |
| author_facet | Nastase, Vivi Merlo, Paola |
| contents | Sentence embeddings from transformer models encode in a fixed length vector much linguistic information. We explore the hypothesis that these embeddings consist of overlapping layers of information that can be separated, and on which specific types of information -- such as information about chunks and their structural and semantic properties -- can be detected. We show that this is the case using a dataset consisting of sentences with known chunk structure, and two linguistic intelligence datasets, solving which relies on detecting chunks and their grammatical number, and respectively, their semantic roles, and through analyses of the performance on the tasks and of the internal representations built during learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_16563 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are there identifiable structural parts in the sentence embedding whole? Nastase, Vivi Merlo, Paola Computation and Language 68T50 I.2.7 Sentence embeddings from transformer models encode in a fixed length vector much linguistic information. We explore the hypothesis that these embeddings consist of overlapping layers of information that can be separated, and on which specific types of information -- such as information about chunks and their structural and semantic properties -- can be detected. We show that this is the case using a dataset consisting of sentences with known chunk structure, and two linguistic intelligence datasets, solving which relies on detecting chunks and their grammatical number, and respectively, their semantic roles, and through analyses of the performance on the tasks and of the internal representations built during learning. |
| title | Are there identifiable structural parts in the sentence embedding whole? |
| topic | Computation and Language 68T50 I.2.7 |
| url | https://arxiv.org/abs/2406.16563 |