VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913161618128896 |
|---|---|
| author | Wu, Yinghao Luo, Zhuoyan Yu, Yiyao Yu, Zhaojian Yang, Yujiu Zhang, Xiao-Ping |
| author_facet | Wu, Yinghao Luo, Zhuoyan Yu, Yiyao Yu, Zhaojian Yang, Yujiu Zhang, Xiao-Ping |
| contents | Despite the remarkable progress achieved by recent efficient methods in accelerating multimodal understanding, they still suffer from noticeable performance degradation. Their emphasis on the high compression ratio of a single visual clue and reliance on the heuristic pruning strategy with coarse attention alignment incurs a bottleneck on the information capacity and density of visual tokens. Addressing this limitation, we propose VEN-VL, a visual ensemble MoE framework for effective and efficient perception following the enrich then compact principle. Specifically, we first enrich the information capacity by unifying the visual representations of different perspectives, and then progressively compact it with adaptive routers in specialized visual experts to enhance the information density. Furthermore, we incorporate the reconstruction ability of vanilla structure via explicit visual supervision, facilitating crucial information preservation. Experimental results demonstrate our superiority in complex visual tasks with few information-condensed tokens, which effectively bridges the gap between performance and efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_25952 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding Wu, Yinghao Luo, Zhuoyan Yu, Yiyao Yu, Zhaojian Yang, Yujiu Zhang, Xiao-Ping Computer Vision and Pattern Recognition Artificial Intelligence Despite the remarkable progress achieved by recent efficient methods in accelerating multimodal understanding, they still suffer from noticeable performance degradation. Their emphasis on the high compression ratio of a single visual clue and reliance on the heuristic pruning strategy with coarse attention alignment incurs a bottleneck on the information capacity and density of visual tokens. Addressing this limitation, we propose VEN-VL, a visual ensemble MoE framework for effective and efficient perception following the enrich then compact principle. Specifically, we first enrich the information capacity by unifying the visual representations of different perspectives, and then progressively compact it with adaptive routers in specialized visual experts to enhance the information density. Furthermore, we incorporate the reconstruction ability of vanilla structure via explicit visual supervision, facilitating crucial information preservation. Experimental results demonstrate our superiority in complex visual tasks with few information-condensed tokens, which effectively bridges the gap between performance and efficiency. |
| title | VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2605.25952 |