CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912539202289664 |
|---|---|
| author | Liu, Xuran Xue, Nan Bao, Rui Sun, Yaping Chen, Zhiyong Tao, Meixia Xu, Xiaodong Cui, Shuguang |
| author_facet | Liu, Xuran Xue, Nan Bao, Rui Sun, Yaping Chen, Zhiyong Tao, Meixia Xu, Xiaodong Cui, Shuguang |
| contents | While deploying large language models on edge devices promises low-latency and privacy-preserving AI services, it is hindered by limited device resources. Although pipeline parallelism facilitates distributed inference, existing approaches often ignore the cold-start latency caused by on-demand model loading. In this paper, we propose a latency-aware scheduling framework that overlaps model loading with computation and communication to minimize total inference latency. Based on device and model parameters, the framework dynamically adjusts layer partitioning and allocation to effectively hide loading time, thereby eliminating as many idle periods as possible. We formulate the problem as a Mixed-Integer Non-Linear Program and design an efficient dynamic programming algorithm to optimize model partitioning and device assignment. Experimental results show that the proposed method significantly reduces cold-start latency compared to baseline strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11287 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems Liu, Xuran Xue, Nan Bao, Rui Sun, Yaping Chen, Zhiyong Tao, Meixia Xu, Xiaodong Cui, Shuguang Information Theory Artificial Intelligence Machine Learning While deploying large language models on edge devices promises low-latency and privacy-preserving AI services, it is hindered by limited device resources. Although pipeline parallelism facilitates distributed inference, existing approaches often ignore the cold-start latency caused by on-demand model loading. In this paper, we propose a latency-aware scheduling framework that overlaps model loading with computation and communication to minimize total inference latency. Based on device and model parameters, the framework dynamically adjusts layer partitioning and allocation to effectively hide loading time, thereby eliminating as many idle periods as possible. We formulate the problem as a Mixed-Integer Non-Linear Program and design an efficient dynamic programming algorithm to optimize model partitioning and device assignment. Experimental results show that the proposed method significantly reduces cold-start latency compared to baseline strategies. |
| title | CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems |
| topic | Information Theory Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2508.11287 |