UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908879048146944 |
|---|---|
| author | Zhao, Yaqi Lin, Wang Zhang, Zijian Yang, Miles Chen, Jingyuan Zhang, Wentao Zhong, Zhao Bo, Liefeng |
| author_facet | Zhao, Yaqi Lin, Wang Zhang, Zijian Yang, Miles Chen, Jingyuan Zhang, Wentao Zhong, Zhao Bo, Liefeng |
| contents | Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10702 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations Zhao, Yaqi Lin, Wang Zhang, Zijian Yang, Miles Chen, Jingyuan Zhang, Wentao Zhong, Zhao Bo, Liefeng Computer Vision and Pattern Recognition Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE. |
| title | UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.10702 |