MoCo: A One-Stop Shop for Model Collaboration Research
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911605284929536 |
|---|---|
| author | Feng, Shangbin Bai, Yuyang Yang, Ziyuan Wang, Yike Tan, Zhaoxuan Yan, Jiajie Lei, Zhenyu Ding, Wenxuan Shi, Weijia Wang, Haojin Qi, Zhenting Jiang, Yuru Wang, Heng Huang, Chengsong Fei, Yu Yao, Jihan Du, Yilun Zettlemoyer, Luke Choi, Yejin Tsvetkov, Yulia |
| author_facet | Feng, Shangbin Bai, Yuyang Yang, Ziyuan Wang, Yike Tan, Zhaoxuan Yan, Jiajie Lei, Zhenyu Ding, Wenxuan Shi, Weijia Wang, Haojin Qi, Zhenting Jiang, Yuru Wang, Heng Huang, Chengsong Fei, Yu Yao, Jihan Du, Yilun Zettlemoyer, Luke Choi, Yejin Tsvetkov, Yulia |
| contents | Advancing beyond single monolithic language models (LMs), recent research increasingly recognizes the importance of model collaboration, where multiple LMs collaborate, compose, and complement each other. Existing research on this topic has mostly been disparate and disconnected, from different research communities, and lacks rigorous comparison. To consolidate existing research and establish model collaboration as a school of thought, we present MoCo: a one-stop Python library of executing, benchmarking, and comparing model collaboration algorithms at scale. MoCo features 26 model collaboration methods, spanning diverse levels of cross-model information exchange such as routing, text, logit, and model parameters. MoCo integrates 25 evaluation datasets spanning reasoning, QA, code, safety, and more, while users could flexibly bring their own data. Extensive experiments with MoCo demonstrate that most collaboration strategies outperform models without collaboration in 61.0% of (model, data) settings on average, with the most effective methods outperforming by up to 25.8%. We further analyze the scaling of model collaboration strategies, the training/inference efficiency of diverse methods, highlight that the collaborative system solves problems where single LMs struggle, and discuss future work in model collaboration, all made possible by MoCo. We envision MoCo as a valuable toolkit to facilitate and turbocharge the quest for an open, modular, decentralized, and collaborative AI future. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_21257 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MoCo: A One-Stop Shop for Model Collaboration Research Feng, Shangbin Bai, Yuyang Yang, Ziyuan Wang, Yike Tan, Zhaoxuan Yan, Jiajie Lei, Zhenyu Ding, Wenxuan Shi, Weijia Wang, Haojin Qi, Zhenting Jiang, Yuru Wang, Heng Huang, Chengsong Fei, Yu Yao, Jihan Du, Yilun Zettlemoyer, Luke Choi, Yejin Tsvetkov, Yulia Computation and Language Advancing beyond single monolithic language models (LMs), recent research increasingly recognizes the importance of model collaboration, where multiple LMs collaborate, compose, and complement each other. Existing research on this topic has mostly been disparate and disconnected, from different research communities, and lacks rigorous comparison. To consolidate existing research and establish model collaboration as a school of thought, we present MoCo: a one-stop Python library of executing, benchmarking, and comparing model collaboration algorithms at scale. MoCo features 26 model collaboration methods, spanning diverse levels of cross-model information exchange such as routing, text, logit, and model parameters. MoCo integrates 25 evaluation datasets spanning reasoning, QA, code, safety, and more, while users could flexibly bring their own data. Extensive experiments with MoCo demonstrate that most collaboration strategies outperform models without collaboration in 61.0% of (model, data) settings on average, with the most effective methods outperforming by up to 25.8%. We further analyze the scaling of model collaboration strategies, the training/inference efficiency of diverse methods, highlight that the collaborative system solves problems where single LMs struggle, and discuss future work in model collaboration, all made possible by MoCo. We envision MoCo as a valuable toolkit to facilitate and turbocharge the quest for an open, modular, decentralized, and collaborative AI future. |
| title | MoCo: A One-Stop Shop for Model Collaboration Research |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.21257 |