A Reparameterized Discrete Diffusion Model for Text Generation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916343187505152 |
|---|---|
| author | Zheng, Lin Yuan, Jianbo Yu, Lei Kong, Lingpeng |
| author_facet | Zheng, Lin Yuan, Jianbo Yu, Lei Kong, Lingpeng |
| contents | This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_05737 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Reparameterized Discrete Diffusion Model for Text Generation Zheng, Lin Yuan, Jianbo Yu, Lei Kong, Lingpeng Computation and Language Machine Learning This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models. |
| title | A Reparameterized Discrete Diffusion Model for Text Generation |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2302.05737 |