Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913565830545408 |
|---|---|
| author | Arkhipkin, Vladimir Vasilev, Viacheslav Filatov, Andrei Pavlov, Igor Agafonova, Julia Gerasimenko, Nikolai Averchenkova, Anna Mironova, Evelina Bukashkin, Anton Kulikov, Konstantin Kuznetsov, Andrey Dimitrov, Denis |
| author_facet | Arkhipkin, Vladimir Vasilev, Viacheslav Filatov, Andrei Pavlov, Igor Agafonova, Julia Gerasimenko, Nikolai Averchenkova, Anna Mironova, Evelina Bukashkin, Anton Kulikov, Konstantin Kuznetsov, Andrey Dimitrov, Denis |
| contents | Text-to-image (T2I) diffusion models are popular for introducing image manipulation methods, such as editing, image fusion, inpainting, etc. At the same time, image-to-video (I2V) and text-to-video (T2V) models are also built on top of T2I models. We present Kandinsky 3, a novel T2I model based on latent diffusion, achieving a high level of quality and photorealism. The key feature of the new architecture is the simplicity and efficiency of its adaptation for many types of generation tasks. We extend the base T2I model for various applications and create a multifunctional generation system that includes text-guided inpainting/outpainting, image fusion, text-image fusion, image variations generation, I2V and T2V generation. We also present a distilled version of the T2I model, evaluating inference in 4 steps of the reverse process without reducing image quality and 3 times faster than the base model. We deployed a user-friendly demo system in which all the features can be tested in the public domain. Additionally, we released the source code and checkpoints for the Kandinsky 3 and extended models. Human evaluations show that Kandinsky 3 demonstrates one of the highest quality scores among open source generation systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_21061 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework Arkhipkin, Vladimir Vasilev, Viacheslav Filatov, Andrei Pavlov, Igor Agafonova, Julia Gerasimenko, Nikolai Averchenkova, Anna Mironova, Evelina Bukashkin, Anton Kulikov, Konstantin Kuznetsov, Andrey Dimitrov, Denis Computer Vision and Pattern Recognition Artificial Intelligence Multimedia Text-to-image (T2I) diffusion models are popular for introducing image manipulation methods, such as editing, image fusion, inpainting, etc. At the same time, image-to-video (I2V) and text-to-video (T2V) models are also built on top of T2I models. We present Kandinsky 3, a novel T2I model based on latent diffusion, achieving a high level of quality and photorealism. The key feature of the new architecture is the simplicity and efficiency of its adaptation for many types of generation tasks. We extend the base T2I model for various applications and create a multifunctional generation system that includes text-guided inpainting/outpainting, image fusion, text-image fusion, image variations generation, I2V and T2V generation. We also present a distilled version of the T2I model, evaluating inference in 4 steps of the reverse process without reducing image quality and 3 times faster than the base model. We deployed a user-friendly demo system in which all the features can be tested in the public domain. Additionally, we released the source code and checkpoints for the Kandinsky 3 and extended models. Human evaluations show that Kandinsky 3 demonstrates one of the highest quality scores among open source generation systems. |
| title | Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2410.21061 |