HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915827698106368 |
|---|---|
| author | Li, Maoliang Chen, Jiayu Zheng, Zihao Li, Ziqian Sun, Xinhao Luo, Guojie Liu, Chenchen Chen, Xiang |
| author_facet | Li, Maoliang Chen, Jiayu Zheng, Zihao Li, Ziqian Sun, Xinhao Luo, Guojie Liu, Chenchen Chen, Xiang |
| contents | With the increasing computational capability of mobile devices, deploying agentic retrieval-augmented generation (RAG) locally on heterogeneous System-on-Chips (SoCs) has become a promising way to enhance LLM-based applications. However, agentic RAG induces multi-stage workflows with heterogeneous models and dynamic execution flow, while mobile SoCs exhibit strong accelerator affinity, shape sensitivity, and shared-memory bandwidth contention, making naive scheduling ineffective. We present HeRo, a heterogeneous-aware framework for low-latency agentic RAG on mobile SoCs. HeRo builds profiling-based performance models for each sub-stage and model-PU configuration, capturing latency, workload shape, and contention-induced slowdown, and leverages them in a lightweight online scheduler that combines shape-aware sub-stage partitioning, criticality-based accelerator mapping, and bandwidth-aware concurrency control. Experiments on commercial mobile devices show that HeRo reduces end-to-end latency by up to $10.94\times$ over existing deployment strategies, enabling practical on-device agentic RAG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01661 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC Li, Maoliang Chen, Jiayu Zheng, Zihao Li, Ziqian Sun, Xinhao Luo, Guojie Liu, Chenchen Chen, Xiang Distributed, Parallel, and Cluster Computing With the increasing computational capability of mobile devices, deploying agentic retrieval-augmented generation (RAG) locally on heterogeneous System-on-Chips (SoCs) has become a promising way to enhance LLM-based applications. However, agentic RAG induces multi-stage workflows with heterogeneous models and dynamic execution flow, while mobile SoCs exhibit strong accelerator affinity, shape sensitivity, and shared-memory bandwidth contention, making naive scheduling ineffective. We present HeRo, a heterogeneous-aware framework for low-latency agentic RAG on mobile SoCs. HeRo builds profiling-based performance models for each sub-stage and model-PU configuration, capturing latency, workload shape, and contention-induced slowdown, and leverages them in a lightweight online scheduler that combines shape-aware sub-stage partitioning, criticality-based accelerator mapping, and bandwidth-aware concurrency control. Experiments on commercial mobile devices show that HeRo reduces end-to-end latency by up to $10.94\times$ over existing deployment strategies, enabling practical on-device agentic RAG. |
| title | HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2603.01661 |