TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914463301500928 |
|---|---|
| author | Ye, Chenhao Zhang, Huaizheng Han, Mingcong Zhong, Baoquan Li, Xiang Chen, Qixiang Zhang, Xinyi Zhang, Weidong Jiang, Kaihua Zhang, Wang Sun, He Xiao, Wencong Arpaci-Dusseau, Andrea C. Arpaci-Dusseau, Remzi H. |
| author_facet | Ye, Chenhao Zhang, Huaizheng Han, Mingcong Zhong, Baoquan Li, Xiang Chen, Qixiang Zhang, Xinyi Zhang, Weidong Jiang, Kaihua Zhang, Wang Sun, He Xiao, Wencong Arpaci-Dusseau, Andrea C. Arpaci-Dusseau, Remzi H. |
| contents | Modern LLM reinforcement learning (RL) workloads require a highly efficient weight transfer system to scale training across heterogeneous computational resources. However, existing weight transfer approaches either fail to provide flexibility for dynamically scaling clusters or incur fundamental data movement overhead, resulting in poor performance.
We introduce Reference-Oriented Storage (ROS), a new storage abstraction for RL weight transfer that exploits the highly replicated model weights in place. ROS presents the illusion that certain versions of the model weights are stored and can be fetched on demand. Underneath, ROS does not physically store any copies of the weights; instead, it tracks the workers that hold these weights on GPUs for inference. Upon request, ROS directly uses them to serve reads. We build TensorHub, a production-quality system that extends the ROS idea with topology-optimized transfer, strong consistency, and fault tolerance. Evaluation shows that TensorHub fully saturates RDMA bandwidth and adapts to three distinct rollout workloads with minimal engineering effort. Specifically, TensorHub reduces total GPU stall time by up to 6.7x for standalone rollouts, accelerates weight update for elastic rollout by 4.8x, and cuts cross-datacenter rollout stall time by 19x. TensorHub has been deployed in production to support cutting-edge RL training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_09107 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training Ye, Chenhao Zhang, Huaizheng Han, Mingcong Zhong, Baoquan Li, Xiang Chen, Qixiang Zhang, Xinyi Zhang, Weidong Jiang, Kaihua Zhang, Wang Sun, He Xiao, Wencong Arpaci-Dusseau, Andrea C. Arpaci-Dusseau, Remzi H. Distributed, Parallel, and Cluster Computing Artificial Intelligence Modern LLM reinforcement learning (RL) workloads require a highly efficient weight transfer system to scale training across heterogeneous computational resources. However, existing weight transfer approaches either fail to provide flexibility for dynamically scaling clusters or incur fundamental data movement overhead, resulting in poor performance. We introduce Reference-Oriented Storage (ROS), a new storage abstraction for RL weight transfer that exploits the highly replicated model weights in place. ROS presents the illusion that certain versions of the model weights are stored and can be fetched on demand. Underneath, ROS does not physically store any copies of the weights; instead, it tracks the workers that hold these weights on GPUs for inference. Upon request, ROS directly uses them to serve reads. We build TensorHub, a production-quality system that extends the ROS idea with topology-optimized transfer, strong consistency, and fault tolerance. Evaluation shows that TensorHub fully saturates RDMA bandwidth and adapts to three distinct rollout workloads with minimal engineering effort. Specifically, TensorHub reduces total GPU stall time by up to 6.7x for standalone rollouts, accelerates weight update for elastic rollout by 4.8x, and cuts cross-datacenter rollout stall time by 19x. TensorHub has been deployed in production to support cutting-edge RL training. |
| title | TensorHub: Scalable and Elastic Weight Transfer for LLM RL Training |
| topic | Distributed, Parallel, and Cluster Computing Artificial Intelligence |
| url | https://arxiv.org/abs/2604.09107 |