Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911953429987328 |
|---|---|
| author | Wu, Zijun Wu, Yongkang Mou, Lili |
| author_facet | Wu, Zijun Wu, Yongkang Mou, Lili |
| contents | Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_01691 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models Wu, Zijun Wu, Yongkang Mou, Lili Computation and Language Artificial Intelligence Prompt tuning in natural language processing (NLP) has become an increasingly popular method for adapting large language models to specific tasks. However, the transferability of these prompts, especially continuous prompts, between different models remains a challenge. In this work, we propose a zero-shot continuous prompt transfer method, where source prompts are encoded into relative space and the corresponding target prompts are searched for transferring to target models. Experimental results confirm the effectiveness of our method, showing that 'task semantics' in continuous prompts can be generalized across various language models. Moreover, we find that combining 'task semantics' from multiple source models can further enhance the generalizability of transfer. |
| title | Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2310.01691 |