Simple Guidance Mechanisms for Discrete Diffusion Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908381744201728 |
|---|---|
| author | Schiff, Yair Sahoo, Subham Sekhar Phung, Hao Wang, Guanghan Boshar, Sam Dalla-torre, Hugo de Almeida, Bernardo P. Rush, Alexander Pierrot, Thomas Kuleshov, Volodymyr |
| author_facet | Schiff, Yair Sahoo, Subham Sekhar Phung, Hao Wang, Guanghan Boshar, Sam Dalla-torre, Hugo de Almeida, Bernardo P. Rush, Alexander Pierrot, Thomas Kuleshov, Volodymyr |
| contents | Diffusion models for continuous data gained widespread adoption owing to their high quality generation and control mechanisms. However, controllable diffusion on discrete data faces challenges given that continuous guidance methods do not directly apply to discrete diffusion. Here, we provide a straightforward derivation of classifier-free and classifier-based guidance for discrete diffusion, as well as a new class of diffusion models that leverage uniform noise and that are more guidable because they can continuously edit their outputs. We improve the quality of these models with a novel continuous-time variational lower bound that yields state-of-the-art performance, especially in settings involving guidance or fast generation. Empirically, we demonstrate that our guidance mechanisms combined with uniform noise diffusion improve controllable generation relative to autoregressive and diffusion baselines on several discrete data domains, including genomic sequences, small molecule design, and discretized image generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_10193 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Simple Guidance Mechanisms for Discrete Diffusion Models Schiff, Yair Sahoo, Subham Sekhar Phung, Hao Wang, Guanghan Boshar, Sam Dalla-torre, Hugo de Almeida, Bernardo P. Rush, Alexander Pierrot, Thomas Kuleshov, Volodymyr Machine Learning Diffusion models for continuous data gained widespread adoption owing to their high quality generation and control mechanisms. However, controllable diffusion on discrete data faces challenges given that continuous guidance methods do not directly apply to discrete diffusion. Here, we provide a straightforward derivation of classifier-free and classifier-based guidance for discrete diffusion, as well as a new class of diffusion models that leverage uniform noise and that are more guidable because they can continuously edit their outputs. We improve the quality of these models with a novel continuous-time variational lower bound that yields state-of-the-art performance, especially in settings involving guidance or fast generation. Empirically, we demonstrate that our guidance mechanisms combined with uniform noise diffusion improve controllable generation relative to autoregressive and diffusion baselines on several discrete data domains, including genomic sequences, small molecule design, and discretized image generation. |
| title | Simple Guidance Mechanisms for Discrete Diffusion Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.10193 |