Distilling Text Style Transfer With Self-Explanation From LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909188951638016 |
|---|---|
| author | Zhang, Chiyu Cai, Honglong Yuezhang Li Wu, Yuexin Hou, Le Abdul-Mageed, Muhammad |
| author_facet | Zhang, Chiyu Cai, Honglong Yuezhang Li Wu, Yuexin Hou, Le Abdul-Mageed, Muhammad |
| contents | Text Style Transfer (TST) seeks to alter the style of text while retaining its core content. Given the constraints of limited parallel datasets for TST, we propose CoTeX, a framework that leverages large language models (LLMs) alongside chain-of-thought (CoT) prompting to facilitate TST. CoTeX distills the complex rewriting and reasoning capabilities of LLMs into more streamlined models capable of working with both non-parallel and parallel data. Through experimentation across four TST datasets, CoTeX is shown to surpass traditional supervised fine-tuning and knowledge distillation methods, particularly in low-resource settings. We conduct a comprehensive evaluation, comparing CoTeX against current unsupervised, supervised, in-context learning (ICL) techniques, and instruction-tuned LLMs. Furthermore, CoTeX distinguishes itself by offering transparent explanations for its style transfer process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01106 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Distilling Text Style Transfer With Self-Explanation From LLMs Zhang, Chiyu Cai, Honglong Yuezhang Li Wu, Yuexin Hou, Le Abdul-Mageed, Muhammad Computation and Language Artificial Intelligence Text Style Transfer (TST) seeks to alter the style of text while retaining its core content. Given the constraints of limited parallel datasets for TST, we propose CoTeX, a framework that leverages large language models (LLMs) alongside chain-of-thought (CoT) prompting to facilitate TST. CoTeX distills the complex rewriting and reasoning capabilities of LLMs into more streamlined models capable of working with both non-parallel and parallel data. Through experimentation across four TST datasets, CoTeX is shown to surpass traditional supervised fine-tuning and knowledge distillation methods, particularly in low-resource settings. We conduct a comprehensive evaluation, comparing CoTeX against current unsupervised, supervised, in-context learning (ICL) techniques, and instruction-tuned LLMs. Furthermore, CoTeX distinguishes itself by offering transparent explanations for its style transfer process. |
| title | Distilling Text Style Transfer With Self-Explanation From LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.01106 |