Ruyi2 Technical Report
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908853436678144 |
|---|---|
| author | Song, Huan Tian, Shuyu Hao, Junyi Xu, Minxiu An, Hongjun Song, Yiliang Shao, Jiawei Li, Xuelong |
| author_facet | Song, Huan Tian, Shuyu Hao, Junyi Xu, Minxiu An, Hongjun Song, Yiliang Shao, Jiawei Li, Xuelong |
| contents | Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies. Building upon the AI Flow framework, we introduce Ruyi2 as an evolution of our adaptive model series designed for efficient variable-depth computation. While early-exit architectures offer a viable efficiency-performance balance, the Ruyi model and existing methods often struggle with optimization complexity and compatibility with large-scale distributed training. To bridge this gap, Ruyi2 introduces a stable "Familial Model" based on Megatron-LM. By using 3D parallel training, it achieves a 2-3 times speedup over Ruyi, while performing comparably to same-sized Qwen3 models. These results confirm that family-based parameter sharing is a highly effective strategy, establishing a new "Train Once, Deploy Many" paradigm and providing a key reference for balancing architectural efficiency with high-performance capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_22543 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ruyi2 Technical Report Song, Huan Tian, Shuyu Hao, Junyi Xu, Minxiu An, Hongjun Song, Yiliang Shao, Jiawei Li, Xuelong Computation and Language Artificial Intelligence Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies. Building upon the AI Flow framework, we introduce Ruyi2 as an evolution of our adaptive model series designed for efficient variable-depth computation. While early-exit architectures offer a viable efficiency-performance balance, the Ruyi model and existing methods often struggle with optimization complexity and compatibility with large-scale distributed training. To bridge this gap, Ruyi2 introduces a stable "Familial Model" based on Megatron-LM. By using 3D parallel training, it achieves a 2-3 times speedup over Ruyi, while performing comparably to same-sized Qwen3 models. These results confirm that family-based parameter sharing is a highly effective strategy, establishing a new "Train Once, Deploy Many" paradigm and providing a key reference for balancing architectural efficiency with high-performance capabilities. |
| title | Ruyi2 Technical Report |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.22543 |