Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.03564 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914367919882240 |
|---|---|
| author | Wu, Shengqiong Wu, Lanhu Bao, Mingyang Xu, Wenhao Zhang, Hanwang Yan, Shuicheng Fei, Hao Chua, Tat-Seng |
| author_facet | Wu, Shengqiong Wu, Lanhu Bao, Mingyang Xu, Wenhao Zhang, Hanwang Yan, Shuicheng Fei, Hao Chua, Tat-Seng |
| contents | Recent advances in large vision models (LVMs) have shifted from modality-specific designs toward unified architectures that jointly process images, videos, and 3D data. However, existing unified LVMs primarily pursue functional integration, while overlooking the deeper goal of cross-vision synergy: the ability to reason over complementary priors across visual modalities. To address this, we present PolyV, a unified LVM that achieves cross-vision synergy at both the architectural and training levels. Architecturally, PolyV adopts a sparse Mixture-of-Experts LVM coordinated by a dynamic modality router, allowing each expert to specialize in modality-specific priors while enabling bidirectional interaction and mutual refinement across modalities. Training-wise, a synergy-aware paradigm combines modality-specific pretraining with coarse-to-fine synergy tuning via knowledge distillation and object-/relation-level alignment. Extensive experiments on 10 benchmarks spanning image, video, and 3D understanding, including synergy-focused datasets requiring spatial or temporal priors, demonstrate that PolyV consistently outperforms existing models, achieving over 10% average improvement over its backbone. Overall, PolyV establishes a unified framework for synesthetic visual reasoning, advancing toward truly synergistic LVMs. Project page: https://sqwu.top/PolyV. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_03564 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Modeling Cross-vision Synergy for Unified Large Vision Model Wu, Shengqiong Wu, Lanhu Bao, Mingyang Xu, Wenhao Zhang, Hanwang Yan, Shuicheng Fei, Hao Chua, Tat-Seng Computer Vision and Pattern Recognition Recent advances in large vision models (LVMs) have shifted from modality-specific designs toward unified architectures that jointly process images, videos, and 3D data. However, existing unified LVMs primarily pursue functional integration, while overlooking the deeper goal of cross-vision synergy: the ability to reason over complementary priors across visual modalities. To address this, we present PolyV, a unified LVM that achieves cross-vision synergy at both the architectural and training levels. Architecturally, PolyV adopts a sparse Mixture-of-Experts LVM coordinated by a dynamic modality router, allowing each expert to specialize in modality-specific priors while enabling bidirectional interaction and mutual refinement across modalities. Training-wise, a synergy-aware paradigm combines modality-specific pretraining with coarse-to-fine synergy tuning via knowledge distillation and object-/relation-level alignment. Extensive experiments on 10 benchmarks spanning image, video, and 3D understanding, including synergy-focused datasets requiring spatial or temporal priors, demonstrate that PolyV consistently outperforms existing models, achieving over 10% average improvement over its backbone. Overall, PolyV establishes a unified framework for synesthetic visual reasoning, advancing toward truly synergistic LVMs. Project page: https://sqwu.top/PolyV. |
| title | Modeling Cross-vision Synergy for Unified Large Vision Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.03564 |