Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.17147 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908602631979008 |
|---|---|
| author | Xia, Linhan Yang, Mingzhan Wang, Jingjing Yan, Ziwei Ren, Yakun Yu, Guo Lei, Kai |
| author_facet | Xia, Linhan Yang, Mingzhan Wang, Jingjing Yan, Ziwei Ren, Yakun Yu, Guo Lei, Kai |
| contents | Transformer-based large language models (LLMs) are increasingly being adopted in networking research to address domain-specific challenges. However, their quadratic time complexity and substantial model sizes often result in significant computational overhead and memory constraints, particularly in resource-constrained environments. Drawing inspiration from the efficiency and performance of the Deepseek-R1 model within the knowledge distillation paradigm, this paper introduces Mamba4Net, a novel cross-architecture distillation framework. Mamba4Net transfers networking-specific knowledge from transformer-based LLMs to student models built on the Mamba architecture, which features linear time complexity. This design substantially enhances computational efficiency compared to the quadratic complexity of transformer-based models, while the reduced model size further minimizes computational demands, improving overall performance and resource utilization. To evaluate its effectiveness, Mamba4Net was tested across three diverse networking tasks: viewport prediction, adaptive bitrate streaming, and cluster job scheduling. Compared to existing methods that do not leverage LLMs, Mamba4Net demonstrates superior task performance. Furthermore, relative to direct applications of transformer-based LLMs, it achieves significant efficiency gains, including a throughput 3.96 times higher and a storage footprint of only 5.48% of that required by previous LLM-based approaches. These results highlight Mamba4Net's potential to enable the cost-effective application of LLM-derived knowledge in networking contexts. The source code is openly available to support further research and development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_17147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mamba4Net: Distilled Hybrid Mamba Large Language Models For Networking Xia, Linhan Yang, Mingzhan Wang, Jingjing Yan, Ziwei Ren, Yakun Yu, Guo Lei, Kai Networking and Internet Architecture Transformer-based large language models (LLMs) are increasingly being adopted in networking research to address domain-specific challenges. However, their quadratic time complexity and substantial model sizes often result in significant computational overhead and memory constraints, particularly in resource-constrained environments. Drawing inspiration from the efficiency and performance of the Deepseek-R1 model within the knowledge distillation paradigm, this paper introduces Mamba4Net, a novel cross-architecture distillation framework. Mamba4Net transfers networking-specific knowledge from transformer-based LLMs to student models built on the Mamba architecture, which features linear time complexity. This design substantially enhances computational efficiency compared to the quadratic complexity of transformer-based models, while the reduced model size further minimizes computational demands, improving overall performance and resource utilization. To evaluate its effectiveness, Mamba4Net was tested across three diverse networking tasks: viewport prediction, adaptive bitrate streaming, and cluster job scheduling. Compared to existing methods that do not leverage LLMs, Mamba4Net demonstrates superior task performance. Furthermore, relative to direct applications of transformer-based LLMs, it achieves significant efficiency gains, including a throughput 3.96 times higher and a storage footprint of only 5.48% of that required by previous LLM-based approaches. These results highlight Mamba4Net's potential to enable the cost-effective application of LLM-derived knowledge in networking contexts. The source code is openly available to support further research and development. |
| title | Mamba4Net: Distilled Hybrid Mamba Large Language Models For Networking |
| topic | Networking and Internet Architecture |
| url | https://arxiv.org/abs/2510.17147 |