MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909659717173248 |
|---|---|
| author | Huang, Zhengxiang Niu, Chaoyue Wang, Zhaode Xue, Jiarui Zhang, Hanming Wang, Yugang Xin, Zewei Jiang, Xiaotang Lv, Chengfei Wu, Fan Chen, Guihai |
| author_facet | Huang, Zhengxiang Niu, Chaoyue Wang, Zhaode Xue, Jiarui Zhang, Hanming Wang, Yugang Xin, Zewei Jiang, Xiaotang Lv, Chengfei Wu, Fan Chen, Guihai |
| contents | As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis shows that the memory-bound LLM decode phase dominates energy use, and yet most existing works focus on accelerating the prefill phase, neglecting energy concerns. We introduce Adaptive Energy-Centric Core Selection (AECS) and integrate it into MNN to create the energy-efficient version, MNN-AECS, the first engine-level system solution without requiring root access or OS modifications for energy-efficient LLM decoding. MNN-AECS is designed to reduce LLM decoding energy while keeping decode speed within an acceptable slowdown threshold by dynamically selecting low-power CPU cores. MNN-AECS is evaluated across 5 Android and 2 iOS devices on 5 popular LLMs of various sizes. Compared to original MNN, MNN-AECS cuts down energy use by 23% without slowdown averaged over all 7 devices and 4 datasets. Against other engines, including llama.cpp, executorch, mllm, and MediaPipe, MNN-AECS delivers 39% to 78% energy saving and 12% to 363% speedup on average. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_19884 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection Huang, Zhengxiang Niu, Chaoyue Wang, Zhaode Xue, Jiarui Zhang, Hanming Wang, Yugang Xin, Zewei Jiang, Xiaotang Lv, Chengfei Wu, Fan Chen, Guihai Operating Systems Artificial Intelligence Performance Software Engineering As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis shows that the memory-bound LLM decode phase dominates energy use, and yet most existing works focus on accelerating the prefill phase, neglecting energy concerns. We introduce Adaptive Energy-Centric Core Selection (AECS) and integrate it into MNN to create the energy-efficient version, MNN-AECS, the first engine-level system solution without requiring root access or OS modifications for energy-efficient LLM decoding. MNN-AECS is designed to reduce LLM decoding energy while keeping decode speed within an acceptable slowdown threshold by dynamically selecting low-power CPU cores. MNN-AECS is evaluated across 5 Android and 2 iOS devices on 5 popular LLMs of various sizes. Compared to original MNN, MNN-AECS cuts down energy use by 23% without slowdown averaged over all 7 devices and 4 datasets. Against other engines, including llama.cpp, executorch, mllm, and MediaPipe, MNN-AECS delivers 39% to 78% energy saving and 12% to 363% speedup on average. |
| title | MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection |
| topic | Operating Systems Artificial Intelligence Performance Software Engineering |
| url | https://arxiv.org/abs/2506.19884 |