BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929659186249728 |
|---|---|
| author | Hao, Shaozhe Liu, Xuantong Qi, Xianbiao Zhao, Shihao Zi, Bojia Xiao, Rong Han, Kai Wong, Kwan-Yee K. |
| author_facet | Hao, Shaozhe Liu, Xuantong Qi, Xianbiao Zhao, Shihao Zi, Bojia Xiao, Rong Han, Kai Wong, Kwan-Yee K. |
| contents | We introduce BiGR, a novel conditional image generation model using compact binary latent codes for generative training, focusing on enhancing both generation and representation capabilities. BiGR is the first conditional generative model that unifies generation and discrimination within the same framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a binary transcoder for binary code prediction. Additionally, we introduce a novel entropy-ordered sampling method to enable efficient image generation. Extensive experiments validate BiGR's superior performance in generation quality, as measured by FID-50k, and representation capabilities, as evidenced by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization across various vision tasks, enabling applications such as image inpainting, outpainting, editing, interpolation, and enrichment, without the need for structural modifications. Our findings suggest that BiGR unifies generative and discriminative tasks effectively, paving the way for further advancements in the field. We further enable BiGR to perform text-to-image generation, showcasing its potential for broader applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_14672 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities Hao, Shaozhe Liu, Xuantong Qi, Xianbiao Zhao, Shihao Zi, Bojia Xiao, Rong Han, Kai Wong, Kwan-Yee K. Computer Vision and Pattern Recognition Artificial Intelligence We introduce BiGR, a novel conditional image generation model using compact binary latent codes for generative training, focusing on enhancing both generation and representation capabilities. BiGR is the first conditional generative model that unifies generation and discrimination within the same framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a binary transcoder for binary code prediction. Additionally, we introduce a novel entropy-ordered sampling method to enable efficient image generation. Extensive experiments validate BiGR's superior performance in generation quality, as measured by FID-50k, and representation capabilities, as evidenced by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization across various vision tasks, enabling applications such as image inpainting, outpainting, editing, interpolation, and enrichment, without the need for structural modifications. Our findings suggest that BiGR unifies generative and discriminative tasks effectively, paving the way for further advancements in the field. We further enable BiGR to perform text-to-image generation, showcasing its potential for broader applications. |
| title | BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2410.14672 |