TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913573425381376 |
|---|---|
| author | Horvitz, Zachary Patel, Ajay Singh, Kanishk Callison-Burch, Chris McKeown, Kathleen Yu, Zhou |
| author_facet | Horvitz, Zachary Patel, Ajay Singh, Kanishk Callison-Burch, Chris McKeown, Kathleen Yu, Zhou |
| contents | The goal of text style transfer is to transform the style of texts while preserving their original meaning, often with only a few examples of the target style. Existing style transfer methods generally rely on the few-shot capabilities of large language models or on complex controllable text generation approaches that are inefficient and underperform on fluency metrics. We introduce TinyStyler, a lightweight but effective approach, which leverages a small language model (800M params) and pre-trained authorship embeddings to perform efficient, few-shot text style transfer. We evaluate on the challenging task of authorship style transfer and find TinyStyler outperforms strong approaches such as GPT-4. We also evaluate TinyStyler's ability to perform text attribute style transfer (formal $\leftrightarrow$ informal) with automatic and human evaluations and find that the approach outperforms recent controllable text generation methods. Our model has been made publicly available at https://huggingface.co/tinystyler/tinystyler . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15586 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings Horvitz, Zachary Patel, Ajay Singh, Kanishk Callison-Burch, Chris McKeown, Kathleen Yu, Zhou Computation and Language The goal of text style transfer is to transform the style of texts while preserving their original meaning, often with only a few examples of the target style. Existing style transfer methods generally rely on the few-shot capabilities of large language models or on complex controllable text generation approaches that are inefficient and underperform on fluency metrics. We introduce TinyStyler, a lightweight but effective approach, which leverages a small language model (800M params) and pre-trained authorship embeddings to perform efficient, few-shot text style transfer. We evaluate on the challenging task of authorship style transfer and find TinyStyler outperforms strong approaches such as GPT-4. We also evaluate TinyStyler's ability to perform text attribute style transfer (formal $\leftrightarrow$ informal) with automatic and human evaluations and find that the approach outperforms recent controllable text generation methods. Our model has been made publicly available at https://huggingface.co/tinystyler/tinystyler . |
| title | TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.15586 |