UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918149885001728 |
|---|---|
| author | Song, Xinyang Wang, Libin Wang, Weining Liu, Shaozhen Zheng, Dandan Chen, Jingdong Li, Qi Sun, Zhenan |
| author_facet | Song, Xinyang Wang, Libin Wang, Weining Liu, Shaozhen Zheng, Dandan Chen, Jingdong Li, Qi Sun, Zhenan |
| contents | The remarkable success of diffusion models in text-to-image generation has sparked growing interest in expanding their capabilities to a variety of multi-modal tasks, including image understanding, manipulation, and perception. These tasks require advanced semantic comprehension across both visual and textual modalities, especially in scenarios involving complex semantic instructions. However, existing approaches often rely heavily on vision-language models (VLMs) or modular designs for semantic guidance, leading to fragmented architectures and computational inefficiency. To address these challenges, we propose UniAlignment, a unified multimodal generation framework within a single diffusion transformer. UniAlignment introduces a dual-stream diffusion training strategy that incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model's cross-modal consistency and instruction-following robustness. Additionally, we present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions. Extensive experiments across multiple tasks and benchmarks demonstrate that UniAlignment outperforms existing baselines, underscoring the significant potential of diffusion models in unified multimodal generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23760 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception Song, Xinyang Wang, Libin Wang, Weining Liu, Shaozhen Zheng, Dandan Chen, Jingdong Li, Qi Sun, Zhenan Computer Vision and Pattern Recognition The remarkable success of diffusion models in text-to-image generation has sparked growing interest in expanding their capabilities to a variety of multi-modal tasks, including image understanding, manipulation, and perception. These tasks require advanced semantic comprehension across both visual and textual modalities, especially in scenarios involving complex semantic instructions. However, existing approaches often rely heavily on vision-language models (VLMs) or modular designs for semantic guidance, leading to fragmented architectures and computational inefficiency. To address these challenges, we propose UniAlignment, a unified multimodal generation framework within a single diffusion transformer. UniAlignment introduces a dual-stream diffusion training strategy that incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model's cross-modal consistency and instruction-following robustness. Additionally, we present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions. Extensive experiments across multiple tasks and benchmarks demonstrate that UniAlignment outperforms existing baselines, underscoring the significant potential of diffusion models in unified multimodal generation. |
| title | UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2509.23760 |