BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910350652211200 |
|---|---|
| author | Zeng, Weihao He, Keqing Wang, Yejie Fu, Dayuan Xu, Weiran |
| author_facet | Zeng, Weihao He, Keqing Wang, Yejie Fu, Dayuan Xu, Weiran |
| contents | Pre-trained language models have been successful in many scenarios. However, their usefulness in task-oriented dialogues is limited due to the intrinsic linguistic differences between general text and task-oriented dialogues. Current task-oriented dialogue pre-training methods rely on a contrastive framework, which faces challenges such as selecting true positives and hard negatives, as well as lacking diversity. In this paper, we propose a novel dialogue pre-training model called BootTOD. It learns task-oriented dialogue representations via a self-bootstrapping framework. Unlike contrastive counterparts, BootTOD aligns context and context+response representations and dismisses the requirements of contrastive pairs. BootTOD also uses multiple appropriate response targets to model the intrinsic one-to-many diversity of human conversations. Experimental results show that BootTOD outperforms strong TOD baselines on diverse downstream dialogue tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01163 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses Zeng, Weihao He, Keqing Wang, Yejie Fu, Dayuan Xu, Weiran Computation and Language Pre-trained language models have been successful in many scenarios. However, their usefulness in task-oriented dialogues is limited due to the intrinsic linguistic differences between general text and task-oriented dialogues. Current task-oriented dialogue pre-training methods rely on a contrastive framework, which faces challenges such as selecting true positives and hard negatives, as well as lacking diversity. In this paper, we propose a novel dialogue pre-training model called BootTOD. It learns task-oriented dialogue representations via a self-bootstrapping framework. Unlike contrastive counterparts, BootTOD aligns context and context+response representations and dismisses the requirements of contrastive pairs. BootTOD also uses multiple appropriate response targets to model the intrinsic one-to-many diversity of human conversations. Experimental results show that BootTOD outperforms strong TOD baselines on diverse downstream dialogue tasks. |
| title | BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.01163 |