UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917866919428096 |
|---|---|
| author | Chen, Xi Zhang, Zhifei Zhang, He Zhou, Yuqian Kim, Soo Ye Liu, Qing Li, Yijun Zhang, Jianming Zhao, Nanxuan Wang, Yilin Ding, Hui Lin, Zhe Zhao, Hengshuang |
| author_facet | Chen, Xi Zhang, Zhifei Zhang, He Zhou, Yuqian Kim, Soo Ye Liu, Qing Li, Yijun Zhang, Jianming Zhao, Nanxuan Wang, Yilin Ding, Hui Lin, Zhe Zhao, Hengshuang |
| contents | We introduce UniReal, a unified framework designed to address various image generation and editing tasks. Existing solutions often vary by tasks, yet share fundamental principles: preserving consistency between inputs and outputs while capturing visual variations. Inspired by recent video generation models that effectively balance consistency and variation across frames, we propose a unifying approach that treats image-level tasks as discontinuous video generation. Specifically, we treat varying numbers of input and output images as frames, enabling seamless support for tasks such as image generation, editing, customization, composition, etc. Although designed for image-level tasks, we leverage videos as a scalable source for universal supervision. UniReal learns world dynamics from large-scale videos, demonstrating advanced capability in handling shadows, reflections, pose variation, and object interaction, while also exhibiting emergent capability for novel applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07774 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics Chen, Xi Zhang, Zhifei Zhang, He Zhou, Yuqian Kim, Soo Ye Liu, Qing Li, Yijun Zhang, Jianming Zhao, Nanxuan Wang, Yilin Ding, Hui Lin, Zhe Zhao, Hengshuang Computer Vision and Pattern Recognition We introduce UniReal, a unified framework designed to address various image generation and editing tasks. Existing solutions often vary by tasks, yet share fundamental principles: preserving consistency between inputs and outputs while capturing visual variations. Inspired by recent video generation models that effectively balance consistency and variation across frames, we propose a unifying approach that treats image-level tasks as discontinuous video generation. Specifically, we treat varying numbers of input and output images as frames, enabling seamless support for tasks such as image generation, editing, customization, composition, etc. Although designed for image-level tasks, we leverage videos as a scalable source for universal supervision. UniReal learns world dynamics from large-scale videos, demonstrating advanced capability in handling shadows, reflections, pose variation, and object interaction, while also exhibiting emergent capability for novel applications. |
| title | UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.07774 |