Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2605.18710 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918509823393792 |
|---|---|
| author | Wang, Yanbo Wang, Yuxuan Chen, Chen Xue, Chunyu Feng, Yu Wu, Anbang Chen, Quan Chen, Yin Weng, Qizhen |
| author_facet | Wang, Yanbo Wang, Yuxuan Chen, Chen Xue, Chunyu Feng, Yu Wu, Anbang Chen, Quan Chen, Yin Weng, Qizhen |
| contents | With the wide adoption of Multimodal Models (MMs) in real-world scenarios, it is significant to efficiently train emerging MMs that exhibit increasingly complex module architectures. For MM deployment, existing works allocate a GPU to only one MM module in a temporal-multiplexing manner; this compromises training efficiency because a single module often fails to achieve high GPU utilization. To improve GPU utilization and enable efficient MM training, we propose deploying MMs in a temporal-spatial multiplexing manner, allowing multiple MM modules to colocate on a GPU with well-controlled resource quotas. In this paper, we propose Apollo, an efficient MM training system that applies temporal-spatial multiplexing. We first develop a flexible and lightweight execution engine that supports MM training with arbitrary resource quotas, and then build a comprehensive and accurate performance model to estimate module execution time under different allocation plans. With the performance model, we further adopt effective heuristics to derive high-quality MM deployment plans efficiently. Testbed experiments confirm that Apollo effectively improves the training efficiency of popular MMs, with a training speedup of up to 1.31x. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18710 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Mosaic: Towards Efficient Training of Multimodal Models with Spatial Resource Multiplexing Wang, Yanbo Wang, Yuxuan Chen, Chen Xue, Chunyu Feng, Yu Wu, Anbang Chen, Quan Chen, Yin Weng, Qizhen Distributed, Parallel, and Cluster Computing With the wide adoption of Multimodal Models (MMs) in real-world scenarios, it is significant to efficiently train emerging MMs that exhibit increasingly complex module architectures. For MM deployment, existing works allocate a GPU to only one MM module in a temporal-multiplexing manner; this compromises training efficiency because a single module often fails to achieve high GPU utilization. To improve GPU utilization and enable efficient MM training, we propose deploying MMs in a temporal-spatial multiplexing manner, allowing multiple MM modules to colocate on a GPU with well-controlled resource quotas. In this paper, we propose Apollo, an efficient MM training system that applies temporal-spatial multiplexing. We first develop a flexible and lightweight execution engine that supports MM training with arbitrary resource quotas, and then build a comprehensive and accurate performance model to estimate module execution time under different allocation plans. With the performance model, we further adopt effective heuristics to derive high-quality MM deployment plans efficiently. Testbed experiments confirm that Apollo effectively improves the training efficiency of popular MMs, with a training speedup of up to 1.31x. |
| title | Mosaic: Towards Efficient Training of Multimodal Models with Spatial Resource Multiplexing |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2605.18710 |