SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910034859917312 |
|---|---|
| author | Yet, Hariz Tam, Nguyen Thanh Ngo, Mao V. Shen, Lim Yi Wei, Lin Park, Jihong Chen, Binbin Quek, Tony Q. S. |
| author_facet | Yet, Hariz Tam, Nguyen Thanh Ngo, Mao V. Shen, Lim Yi Wei, Lin Park, Jihong Chen, Binbin Quek, Tony Q. S. |
| contents | Embodied AI requires sub-second inference near the Radio Access Network (RAN), but deployments span heterogeneous tiers (on-device, RAN-edge, cloud) and must not disrupt real-time baseband processing. We report measurements from a 5G Standalone (SA) AI-RAN testbed using a fixed baseline policy for repeatability. The setup includes an on-device tier, a three-node RAN-edge cluster co-hosting a containerized 5G RAN, and a cloud tier. We find that on-device execution remains multi-second and fails to meet sub-second budgets. At the RAN edge, SLA feasibility is primarily determined by model variant choice: quantized models concentrate below 0.5\,s, while unquantized and some larger quantized models incur deadline misses due to stalls and queuing. In the cloud tier, meeting a 0.5\,s deadline is challenging on the measured WAN path (up to 32.9\% of requests complete within 0.5\,s), but all evaluated variants meet a 1.0\,s deadline (100\% within 1.0\,s). Under saturated downlink traffic and up to $N{=}20$ concurrent inference clients, Multi-Instance GPU (MIG) isolation preserves baseband timing-health proxies, supporting safe co-location under fixed partitioning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23722 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud Yet, Hariz Tam, Nguyen Thanh Ngo, Mao V. Shen, Lim Yi Wei, Lin Park, Jihong Chen, Binbin Quek, Tony Q. S. Networking and Internet Architecture Artificial Intelligence Embodied AI requires sub-second inference near the Radio Access Network (RAN), but deployments span heterogeneous tiers (on-device, RAN-edge, cloud) and must not disrupt real-time baseband processing. We report measurements from a 5G Standalone (SA) AI-RAN testbed using a fixed baseline policy for repeatability. The setup includes an on-device tier, a three-node RAN-edge cluster co-hosting a containerized 5G RAN, and a cloud tier. We find that on-device execution remains multi-second and fails to meet sub-second budgets. At the RAN edge, SLA feasibility is primarily determined by model variant choice: quantized models concentrate below 0.5\,s, while unquantized and some larger quantized models incur deadline misses due to stalls and queuing. In the cloud tier, meeting a 0.5\,s deadline is challenging on the measured WAN path (up to 32.9\% of requests complete within 0.5\,s), but all evaluated variants meet a 1.0\,s deadline (100\% within 1.0\,s). Under saturated downlink traffic and up to $N{=}20$ concurrent inference clients, Multi-Instance GPU (MIG) isolation preserves baseband timing-health proxies, supporting safe co-location under fixed partitioning. |
| title | SLA-Aware Distributed LLM Inference Across Device-RAN-Cloud |
| topic | Networking and Internet Architecture Artificial Intelligence |
| url | https://arxiv.org/abs/2602.23722 |