Continuous Prompt Generation from Linear Combination of Discrete Prompt Embeddings
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914678529064960 |
|---|---|
| author | Passigan, Pascal Yohannes, Kidus Pereira, Joshua |
| author_facet | Passigan, Pascal Yohannes, Kidus Pereira, Joshua |
| contents | The wayward quality of continuous prompts stresses the importance of their interpretability as unexpected and unpredictable behaviors appear following training, especially in the context of large language models automating people-sensitive tasks such as resume screening. In this paper we present a novel method of constructing continuous prompts via discrete prompt embeddings and evaluate improvements to continuous prompt interpretability and inference accuracy. For a set of manually designed discrete prompts $\mathcal{D}$, which we tokenize and embed each into tensor form, we train a model to predict the weights such that the linear combinations of those prompts correspond to higher performance on natural language understanding tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_10323 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Continuous Prompt Generation from Linear Combination of Discrete Prompt Embeddings Passigan, Pascal Yohannes, Kidus Pereira, Joshua Computation and Language The wayward quality of continuous prompts stresses the importance of their interpretability as unexpected and unpredictable behaviors appear following training, especially in the context of large language models automating people-sensitive tasks such as resume screening. In this paper we present a novel method of constructing continuous prompts via discrete prompt embeddings and evaluate improvements to continuous prompt interpretability and inference accuracy. For a set of manually designed discrete prompts $\mathcal{D}$, which we tokenize and embed each into tensor form, we train a model to predict the weights such that the linear combinations of those prompts correspond to higher performance on natural language understanding tasks. |
| title | Continuous Prompt Generation from Linear Combination of Discrete Prompt Embeddings |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2312.10323 |