Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.19009 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914824357675008 |
|---|---|
| author | Liu, Guangyi Wang, Yu Feng, Zeyu Wu, Qiyu Tang, Liping Gao, Yuan Li, Zhen Cui, Shuguang McAuley, Julian Yang, Zichao Xing, Eric P. Hu, Zhiting |
| author_facet | Liu, Guangyi Wang, Yu Feng, Zeyu Wu, Qiyu Tang, Liping Gao, Yuan Li, Zhen Cui, Shuguang McAuley, Julian Yang, Zichao Xing, Eric P. Hu, Zhiting |
| contents | The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images. Existing model families, like variational autoencoders (VAEs), generative adversarial networks (GANs), autoregressive models, and (latent) diffusion models, generally excel in specific capabilities and data types but fall short in others. We introduce Generalized Encoding-Decoding Diffusion Probabilistic Models (EDDPMs) which integrate the core capabilities for broad applicability and enhanced performance. EDDPMs generalize the Gaussian noising-denoising in standard diffusion by introducing parameterized encoding-decoding. Crucially, EDDPMs are compatible with the well-established diffusion model objective and training recipes, allowing effective learning of the encoder-decoder parameters jointly with diffusion. By choosing appropriate encoder/decoder (e.g., large language models), EDDPMs naturally apply to different data types. Extensive experiments on text, proteins, and images demonstrate the flexibility to handle diverse data and tasks and the strong improvement over various existing models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_19009 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding Liu, Guangyi Wang, Yu Feng, Zeyu Wu, Qiyu Tang, Liping Gao, Yuan Li, Zhen Cui, Shuguang McAuley, Julian Yang, Zichao Xing, Eric P. Hu, Zhiting Machine Learning Artificial Intelligence The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images. Existing model families, like variational autoencoders (VAEs), generative adversarial networks (GANs), autoregressive models, and (latent) diffusion models, generally excel in specific capabilities and data types but fall short in others. We introduce Generalized Encoding-Decoding Diffusion Probabilistic Models (EDDPMs) which integrate the core capabilities for broad applicability and enhanced performance. EDDPMs generalize the Gaussian noising-denoising in standard diffusion by introducing parameterized encoding-decoding. Crucially, EDDPMs are compatible with the well-established diffusion model objective and training recipes, allowing effective learning of the encoder-decoder parameters jointly with diffusion. By choosing appropriate encoder/decoder (e.g., large language models), EDDPMs naturally apply to different data types. Extensive experiments on text, proteins, and images demonstrate the flexibility to handle diverse data and tasks and the strong improvement over various existing models. |
| title | Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.19009 |