Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.03147 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908413095575552 |
|---|---|
| author | Lin, Bin Li, Zongjian Cheng, Xinhua Niu, Yuwei Ye, Yang He, Xianyi Yuan, Shenghai Yu, Wangbo Wang, Shaodong Ge, Yunyang Pang, Yatian Yuan, Li |
| author_facet | Lin, Bin Li, Zongjian Cheng, Xinhua Niu, Yuwei Ye, Yang He, Xianyi Yuan, Shenghai Yu, Wangbo Wang, Shaodong Ge, Yunyang Pang, Yatian Yuan, Li |
| contents | Although existing unified models achieve strong performance in vision-language understanding and text-to-image generation, they remain limited in addressing image perception and manipulation -- capabilities increasingly demanded in practical applications. Recently, OpenAI introduced the powerful GPT-4o-Image model, which showcases advanced capabilities in comprehensive image perception and manipulation, sparking widespread interest. Through carefully designed experiments, we observe that GPT-4o-Image likely relies on semantic encoders rather than VAEs for feature extraction, despite VAEs being commonly regarded as crucial for image manipulation tasks. Inspired by this insight, we propose UniWorld-V1, a unified generative framework built upon semantic features extracted from powerful multimodal large language models and contrastive semantic encoders. Using only 2.7M training data, UniWorld-V1 achieves impressive performance across diverse tasks, including image understanding, generation, manipulation, and perception. We fully open-source the UniWorld-V1 framework, including model weights, training and evaluation scripts, and datasets to promote reproducibility and further research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_03147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation Lin, Bin Li, Zongjian Cheng, Xinhua Niu, Yuwei Ye, Yang He, Xianyi Yuan, Shenghai Yu, Wangbo Wang, Shaodong Ge, Yunyang Pang, Yatian Yuan, Li Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Although existing unified models achieve strong performance in vision-language understanding and text-to-image generation, they remain limited in addressing image perception and manipulation -- capabilities increasingly demanded in practical applications. Recently, OpenAI introduced the powerful GPT-4o-Image model, which showcases advanced capabilities in comprehensive image perception and manipulation, sparking widespread interest. Through carefully designed experiments, we observe that GPT-4o-Image likely relies on semantic encoders rather than VAEs for feature extraction, despite VAEs being commonly regarded as crucial for image manipulation tasks. Inspired by this insight, we propose UniWorld-V1, a unified generative framework built upon semantic features extracted from powerful multimodal large language models and contrastive semantic encoders. Using only 2.7M training data, UniWorld-V1 achieves impressive performance across diverse tasks, including image understanding, generation, manipulation, and perception. We fully open-source the UniWorld-V1 framework, including model weights, training and evaluation scripts, and datasets to promote reproducibility and further research. |
| title | UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2506.03147 |