Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2605.13779 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911719405649920 |
|---|---|
| author | Lab, Mind : Cao, Song Cao, Vic Chen, Andrew Chen, Kaijie Cheng, Cleon Chiang, Steven Fan, Kaixuan Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Ho, Nolan Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Qiuyu Kong, Fancy Lei, Andrew Lei, Kyrie Li, Alexy Li, Lucian Li, Ray Li, Theo Li, Zhihui Lin, Jiayi Liu, Kairus Liu, Kieran Liu, Logan Liu, Xiang Lu, Irvine Luo, Maeve Lv, Runze Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yang, Rio Yao, Maxwell Ye, Carrie Ye, Regis Ye, Wenlin Ying, Josh Zeng, Danney Zhan, Yuhan Zhang, Anya Zhang, Di Zhang, Ruijia Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Changhai Zhou, Yuhua Zhu, Xinyue Zhuang, Murphy |
| author_facet | Lab, Mind : Cao, Song Cao, Vic Chen, Andrew Chen, Kaijie Cheng, Cleon Chiang, Steven Fan, Kaixuan Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Ho, Nolan Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Qiuyu Kong, Fancy Lei, Andrew Lei, Kyrie Li, Alexy Li, Lucian Li, Ray Li, Theo Li, Zhihui Lin, Jiayi Liu, Kairus Liu, Kieran Liu, Logan Liu, Xiang Lu, Irvine Luo, Maeve Lv, Runze Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yang, Rio Yao, Maxwell Ye, Carrie Ye, Regis Ye, Wenlin Ying, Josh Zeng, Danney Zhan, Yuhan Zhang, Anya Zhang, Di Zhang, Ruijia Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Changhai Zhou, Yuhua Zhu, Xinyue Zhuang, Murphy |
| contents | We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained policies are produced over a small number of expensive base-model deployments. Instead of materializing each policy as a merged full checkpoint, MinT keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving, and rollback, hiding distributed training, serving, scheduling, and data movement behind a service interface. MinT scales this path along three axes. Scale Up extends LoRA RL to frontier-scale dense and MoE architectures, including MLA and DSA attention paths, with training and serving validated beyond 1T total parameters. Scale Down moves only the exported LoRA adapter, which can be under 1% of base-model size in rank-1 settings; adapter-only handoff reduces the measured step by 18.3x on a 4B dense model and 2.85x on a 30B MoE, while concurrent multi-policy GRPO shortens wall time by 1.77x and 1.45x without raising peak memory. Scale Out separates durable policy addressability from CPU/GPU working sets: a tensor-parallel deployment supports 10^6-scale addressable catalogs (measured single-engine sweeps through 100K) and thousand-adapter active waves at cluster scale, with cold loading treated as scheduled service work and packed MoE LoRA tensors improving live engine loading by 8.5-8.7x. MinT thus manages million-scale LoRA policy catalogs while training and serving selected adapter revisions over shared 1T-class base models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_13779 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MinT: Managed Infrastructure for Training and Serving Millions of LLMs Lab, Mind : Cao, Song Cao, Vic Chen, Andrew Chen, Kaijie Cheng, Cleon Chiang, Steven Fan, Kaixuan Feng, Hera Feng, Huan Fu, Arthur Gao, Jun Gu, Hongquan Guan, Aaron Ho, Nolan Hong, Mutian Hou, Hailee Hua, Peixuan Huang, Charles Jiang, Miles Jiang, Nora Jiang, Yuyi Jin, Qiuyu Kong, Fancy Lei, Andrew Lei, Kyrie Li, Alexy Li, Lucian Li, Ray Li, Theo Li, Zhihui Lin, Jiayi Liu, Kairus Liu, Kieran Liu, Logan Liu, Xiang Lu, Irvine Luo, Maeve Lv, Runze Ma, Pony Niu, Verity Qiu, Anson Wang, Vincent Yang, Rio Yao, Maxwell Ye, Carrie Ye, Regis Ye, Wenlin Ying, Josh Zeng, Danney Zhan, Yuhan Zhang, Anya Zhang, Di Zhang, Ruijia Zhang, Sueky Zhang, Ya Zhao, Wei Zhou, Ada Zhou, Changhai Zhou, Yuhua Zhu, Xinyue Zhuang, Murphy Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing We present MindLab Toolkit (MinT), a managed infrastructure system for Low-Rank Adaptation (LoRA) post-training and online serving. MinT targets a setting where many trained policies are produced over a small number of expensive base-model deployments. Instead of materializing each policy as a merged full checkpoint, MinT keeps the base model resident and moves exported LoRA adapter revisions through rollout, update, export, evaluation, serving, and rollback, hiding distributed training, serving, scheduling, and data movement behind a service interface. MinT scales this path along three axes. Scale Up extends LoRA RL to frontier-scale dense and MoE architectures, including MLA and DSA attention paths, with training and serving validated beyond 1T total parameters. Scale Down moves only the exported LoRA adapter, which can be under 1% of base-model size in rank-1 settings; adapter-only handoff reduces the measured step by 18.3x on a 4B dense model and 2.85x on a 30B MoE, while concurrent multi-policy GRPO shortens wall time by 1.77x and 1.45x without raising peak memory. Scale Out separates durable policy addressability from CPU/GPU working sets: a tensor-parallel deployment supports 10^6-scale addressable catalogs (measured single-engine sweeps through 100K) and thousand-adapter active waves at cluster scale, with cold loading treated as scheduled service work and packed MoE LoRA tensors improving live engine loading by 8.5-8.7x. MinT thus manages million-scale LoRA policy catalogs while training and serving selected adapter revisions over shared 1T-class base models. |
| title | MinT: Managed Infrastructure for Training and Serving Millions of LLMs |
| topic | Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2605.13779 |