Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2309.13426 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913198448312320 |
|---|---|
| author | Zhang, Yang Bartley, Travis M. Graterol-Fuenmayor, Mariana Lavrukhin, Vitaly Bakhturina, Evelina Ginsburg, Boris |
| author_facet | Zhang, Yang Bartley, Travis M. Graterol-Fuenmayor, Mariana Lavrukhin, Vitaly Bakhturina, Evelina Ginsburg, Boris |
| contents | Text normalization - the conversion of text from written to spoken form - is traditionally assumed to be an ill-formed task for language models. In this work, we argue otherwise. We empirically show the capacity of Large-Language Models (LLM) for text normalization in few-shot scenarios. Combining self-consistency reasoning with linguistic-informed prompt engineering, we find LLM based text normalization to achieve error rates around 40\% lower than top normalization systems. Further, upon error analysis, we note key limitations in the conventional design of text normalization tasks. We create a new taxonomy of text normalization errors and apply it to results from GPT-3.5-Turbo and GPT-4.0. Through this new framework, we can identify strengths and weaknesses of GPT-based TN, opening opportunities for future work. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_13426 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Chat About Boring Problems: Studying GPT-based text normalization Zhang, Yang Bartley, Travis M. Graterol-Fuenmayor, Mariana Lavrukhin, Vitaly Bakhturina, Evelina Ginsburg, Boris Computation and Language Artificial Intelligence Text normalization - the conversion of text from written to spoken form - is traditionally assumed to be an ill-formed task for language models. In this work, we argue otherwise. We empirically show the capacity of Large-Language Models (LLM) for text normalization in few-shot scenarios. Combining self-consistency reasoning with linguistic-informed prompt engineering, we find LLM based text normalization to achieve error rates around 40\% lower than top normalization systems. Further, upon error analysis, we note key limitations in the conventional design of text normalization tasks. We create a new taxonomy of text normalization errors and apply it to results from GPT-3.5-Turbo and GPT-4.0. Through this new framework, we can identify strengths and weaknesses of GPT-based TN, opening opportunities for future work. |
| title | A Chat About Boring Problems: Studying GPT-based text normalization |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2309.13426 |