Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916276192935936 |
|---|---|
| author | Guo, Ruohao Xu, Wei Ritter, Alan |
| author_facet | Guo, Ruohao Xu, Wei Ritter, Alan |
| contents | Language style is often used by writers to convey their intentions, identities, and mastery of language. In this paper, we show that current large language models struggle to capture some language styles without fine-tuning. To address this challenge, we investigate whether LLMs can be meta-trained based on representative lexicons to recognize new styles they have not been fine-tuned on. Experiments on 13 established style classification tasks, as well as 63 novel tasks generated using LLMs, demonstrate that meta-training with style lexicons consistently improves zero-shot transfer across styles. We release the code and data at http://github.com/octaviaguo/Style-LLM . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14592 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding Guo, Ruohao Xu, Wei Ritter, Alan Computation and Language Machine Learning Language style is often used by writers to convey their intentions, identities, and mastery of language. In this paper, we show that current large language models struggle to capture some language styles without fine-tuning. To address this challenge, we investigate whether LLMs can be meta-trained based on representative lexicons to recognize new styles they have not been fine-tuned on. Experiments on 13 established style classification tasks, as well as 63 novel tasks generated using LLMs, demonstrate that meta-training with style lexicons consistently improves zero-shot transfer across styles. We release the code and data at http://github.com/octaviaguo/Style-LLM . |
| title | Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2305.14592 |