Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2306.04357 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910385036066816 |
|---|---|
| author | Su, Zhenpeng Wu, Xing Zhou, Wei Ma, Guangyuan Hu, Songlin |
| author_facet | Su, Zhenpeng Wu, Xing Zhou, Wei Ma, Guangyuan Hu, Songlin |
| contents | Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus on post-training and fine-tuning tailored for cross-encoders. However, there are no post-training methods tailored for dense encoders in dialogue response selection. We argue that when the current language model, based on dense dialogue systems (such as BERT), is employed as a dense encoder, it separately encodes dialogue context and response, leading to a struggle to achieve the alignment of both representations. Thus, we propose Dial-MAE (Dialogue Contextual Masking Auto-Encoder), a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Dial-MAE uses an asymmetric encoder-decoder architecture to compress the dialogue semantics into dense vectors, which achieves better alignment between the features of the dialogue context and response. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_04357 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems Su, Zhenpeng Wu, Xing Zhou, Wei Ma, Guangyuan Hu, Songlin Computation and Language Artificial Intelligence Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus on post-training and fine-tuning tailored for cross-encoders. However, there are no post-training methods tailored for dense encoders in dialogue response selection. We argue that when the current language model, based on dense dialogue systems (such as BERT), is employed as a dense encoder, it separately encodes dialogue context and response, leading to a struggle to achieve the alignment of both representations. Thus, we propose Dial-MAE (Dialogue Contextual Masking Auto-Encoder), a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Dial-MAE uses an asymmetric encoder-decoder architecture to compress the dialogue semantics into dense vectors, which achieves better alignment between the features of the dialogue context and response. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks. |
| title | Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2306.04357 |