HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913938986237952 |
|---|---|
| author | Hu, Zhengding Murthy, Vibha Pan, Zaifeng Li, Wanlu Fang, Xiaoyi Ding, Yufei Wang, Yuke |
| author_facet | Hu, Zhengding Murthy, Vibha Pan, Zaifeng Li, Wanlu Fang, Xiaoyi Ding, Yufei Wang, Yuke |
| contents | This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request patterns complicate efficient execution. We present HedraRAG, a runtime system built on a graph-based abstraction that exposes optimization opportunities across stage-level parallelism, intra-request similarity, and inter-request skewness. These opportunities are realized through dynamic graph transformations, such as node splitting, reordering, edge addition, and dependency rewiring, applied to wavefronts of subgraphs spanning concurrent requests. The resulting execution plans are mapped onto hybrid CPU-GPU pipelines to improve resource utilization and reduce latency. Evaluations across a wide range of RAG workflows demonstrate speedups exceeding 1.5x and reaching up to 5x over existing frameworks, showcasing the effectiveness of coordinated generation and retrieval in serving environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09138 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving Hu, Zhengding Murthy, Vibha Pan, Zaifeng Li, Wanlu Fang, Xiaoyi Ding, Yufei Wang, Yuke Databases Machine Learning This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request patterns complicate efficient execution. We present HedraRAG, a runtime system built on a graph-based abstraction that exposes optimization opportunities across stage-level parallelism, intra-request similarity, and inter-request skewness. These opportunities are realized through dynamic graph transformations, such as node splitting, reordering, edge addition, and dependency rewiring, applied to wavefronts of subgraphs spanning concurrent requests. The resulting execution plans are mapped onto hybrid CPU-GPU pipelines to improve resource utilization and reduce latency. Evaluations across a wide range of RAG workflows demonstrate speedups exceeding 1.5x and reaching up to 5x over existing frameworks, showcasing the effectiveness of coordinated generation and retrieval in serving environments. |
| title | HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving |
| topic | Databases Machine Learning |
| url | https://arxiv.org/abs/2507.09138 |