DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929684904673280 |
|---|---|
| author | Gu, Jiatao Wang, Yuyang Zhang, Yizhe Zhang, Qihang Zhang, Dinghuai Jaitly, Navdeep Susskind, Josh Zhai, Shuangfei |
| author_facet | Gu, Jiatao Wang, Yuyang Zhang, Yizhe Zhang, Qihang Zhang, Dinghuai Jaitly, Navdeep Susskind, Josh Zhai, Shuangfei |
| contents | Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that the Markovian property limits the model's ability to fully utilize the generation trajectory, leading to inefficiencies during training and inference. In this paper, we propose DART, a transformer-based model that unifies autoregressive (AR) and diffusion within a non-Markovian framework. DART iteratively denoises image patches spatially and spectrally using an AR model that has the same architecture as standard language models. DART does not rely on image quantization, which enables more effective image modeling while maintaining flexibility. Furthermore, DART seamlessly trains with both text and image data in a unified model. Our approach demonstrates competitive performance on class-conditioned and text-to-image generation tasks, offering a scalable, efficient alternative to traditional diffusion models. Through this unified framework, DART sets a new benchmark for scalable, high-quality image synthesis. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08159 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation Gu, Jiatao Wang, Yuyang Zhang, Yizhe Zhang, Qihang Zhang, Dinghuai Jaitly, Navdeep Susskind, Josh Zhai, Shuangfei Computer Vision and Pattern Recognition Machine Learning Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that the Markovian property limits the model's ability to fully utilize the generation trajectory, leading to inefficiencies during training and inference. In this paper, we propose DART, a transformer-based model that unifies autoregressive (AR) and diffusion within a non-Markovian framework. DART iteratively denoises image patches spatially and spectrally using an AR model that has the same architecture as standard language models. DART does not rely on image quantization, which enables more effective image modeling while maintaining flexibility. Furthermore, DART seamlessly trains with both text and image data in a unified model. Our approach demonstrates competitive performance on class-conditioned and text-to-image generation tasks, offering a scalable, efficient alternative to traditional diffusion models. Through this unified framework, DART sets a new benchmark for scalable, high-quality image synthesis. |
| title | DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2410.08159 |