Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2307.07047 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929248449593344 |
|---|---|
| author | Lu, Bo-Ru Haduong, Nikita Lee, Chia-Hsuan Wu, Zeqiu Cheng, Hao Koester, Paul Utke, Jean Yu, Tao Smith, Noah A. Ostendorf, Mari |
| author_facet | Lu, Bo-Ru Haduong, Nikita Lee, Chia-Hsuan Wu, Zeqiu Cheng, Hao Koester, Paul Utke, Jean Yu, Tao Smith, Noah A. Ostendorf, Mari |
| contents | The capabilities of pretrained language models have opened opportunities to explore new application areas, but applications involving human-human interaction are limited by the fact that most data is protected from public release for privacy reasons. Problem-solving human dialogues in real applications can be much more complex than existing Wizard-of-Oz collections, preventing successful domain transfer. To support information extraction (IE) for a private call center dataset, we introduce a human-in-the-loop dialogue generation framework capable of synthesizing realistic dialogues. In IE experiments with auto insurance call center dialogues, we observe 25\% relative improvement in $F_1$ after augmenting a small set of real human conversations with synthetic data. We release code and our synthetic dataset to illustrate the complexity of real-world call center conversations and encourage development of complex dialogue datasets that are more representative of natural data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_07047 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues? Lu, Bo-Ru Haduong, Nikita Lee, Chia-Hsuan Wu, Zeqiu Cheng, Hao Koester, Paul Utke, Jean Yu, Tao Smith, Noah A. Ostendorf, Mari Computation and Language The capabilities of pretrained language models have opened opportunities to explore new application areas, but applications involving human-human interaction are limited by the fact that most data is protected from public release for privacy reasons. Problem-solving human dialogues in real applications can be much more complex than existing Wizard-of-Oz collections, preventing successful domain transfer. To support information extraction (IE) for a private call center dataset, we introduce a human-in-the-loop dialogue generation framework capable of synthesizing realistic dialogues. In IE experiments with auto insurance call center dialogues, we observe 25\% relative improvement in $F_1$ after augmenting a small set of real human conversations with synthetic data. We release code and our synthetic dataset to illustrate the complexity of real-world call center conversations and encourage development of complex dialogue datasets that are more representative of natural data. |
| title | Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2307.07047 |