Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.18010 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909505138196480 |
|---|---|
| author | You, Feiran Du, Hongyang Huang, Kaibin Jamalipour, Abbas |
| author_facet | You, Feiran Du, Hongyang Huang, Kaibin Jamalipour, Abbas |
| contents | Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, leading to their increasing deployment in wireless networks for a wide variety of user services. However, the growing longer prompt setting highlights the crucial issue of computational resource demands and huge communication load. To address this challenge, we propose Joint Power and Prompt Optimization (JPPO), a framework that combines Small Language Model (SLM)-based prompt compression with wireless power allocation optimization. By deploying SLM at user devices for prompt compression and employing Deep Reinforcement Learning for joint optimization of compression ratio and transmission power, JPPO effectively balances service quality with resource efficiency. Experimental results demonstrate that our framework achieves high service fidelity and low bit error rates while optimizing power usage in wireless LLM services. The system reduces response time by about 17%, with the improvement varying based on the length of the original prompt. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18010 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | JPPO: Joint Power and Prompt Optimization for Accelerated Large Language Model Services You, Feiran Du, Hongyang Huang, Kaibin Jamalipour, Abbas Audio and Speech Processing Computation and Language Sound Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, leading to their increasing deployment in wireless networks for a wide variety of user services. However, the growing longer prompt setting highlights the crucial issue of computational resource demands and huge communication load. To address this challenge, we propose Joint Power and Prompt Optimization (JPPO), a framework that combines Small Language Model (SLM)-based prompt compression with wireless power allocation optimization. By deploying SLM at user devices for prompt compression and employing Deep Reinforcement Learning for joint optimization of compression ratio and transmission power, JPPO effectively balances service quality with resource efficiency. Experimental results demonstrate that our framework achieves high service fidelity and low bit error rates while optimizing power usage in wireless LLM services. The system reduces response time by about 17%, with the improvement varying based on the length of the original prompt. |
| title | JPPO: Joint Power and Prompt Optimization for Accelerated Large Language Model Services |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2411.18010 |