Dialogue Language Model with Large-Scale Persona Data Engineering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929719873634304 |
|---|---|
| author | Hong, Mengze Zhang, Chen Jason Chen, Chaotao Lian, Rongzhong Jiang, Di |
| author_facet | Hong, Mengze Zhang, Chen Jason Chen, Chaotao Lian, Rongzhong Jiang, Di |
| contents | Maintaining persona consistency is paramount in the application of open-domain dialogue systems, as exemplified by models like ChatGPT. Despite significant advancements, the limited scale and diversity of current persona dialogue datasets remain challenges to achieving robust persona-consistent dialogue models. In this study, drawing inspiration from the success of large-scale pre-training, we introduce PPDS, an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency. Specifically, we present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets. Additionally, we unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset. Both quantitative and human evaluations consistently highlight the superior response quality and persona consistency of our proposed model, underscoring its effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_09034 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Dialogue Language Model with Large-Scale Persona Data Engineering Hong, Mengze Zhang, Chen Jason Chen, Chaotao Lian, Rongzhong Jiang, Di Computation and Language Human-Computer Interaction Maintaining persona consistency is paramount in the application of open-domain dialogue systems, as exemplified by models like ChatGPT. Despite significant advancements, the limited scale and diversity of current persona dialogue datasets remain challenges to achieving robust persona-consistent dialogue models. In this study, drawing inspiration from the success of large-scale pre-training, we introduce PPDS, an open-domain persona dialogue system that employs extensive generative pre-training on a persona dialogue dataset to enhance persona consistency. Specifically, we present a persona extraction model designed to autonomously and precisely generate vast persona dialogue datasets. Additionally, we unveil a pioneering persona augmentation technique to address the invalid persona bias inherent in the constructed dataset. Both quantitative and human evaluations consistently highlight the superior response quality and persona consistency of our proposed model, underscoring its effectiveness. |
| title | Dialogue Language Model with Large-Scale Persona Data Engineering |
| topic | Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2412.09034 |