Pancake: Hierarchical Memory System for Multi-Agent LLM Serving
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911467310153728 |
|---|---|
| author | Hu, Zhengding Pan, Zaifeng Kaur, Prabhleen Murthy, Vibha Yu, Zhongkai Guan, Yue Wang, Zhen Swanson, Steven Ding, Yufei |
| author_facet | Hu, Zhengding Pan, Zaifeng Kaur, Prabhleen Murthy, Vibha Yu, Zhongkai Guan, Yue Wang, Zhen Swanson, Steven Ding, Yufei |
| contents | In this work, we identify and address the core challenges of agentic memory management in LLM serving, where large-scale storage, frequent updates, and multiple coexisting agents jointly introduce complex and high-cost approximate nearest neighbor (ANN) searching problems. We present Pancake, a multi-tier agentic memory system that unifies three key techniques: (i) multi-level index caching for single agents, (ii) coordinated index management across multiple agents, and (iii) collaborative GPU-CPU acceleration. Pancake exposes easy-to-use interface that can be integrated into memory-based agents like Mem-GPT, and is compatible with agentic frameworks such as LangChain and LlamaIndex. Experiments on realistic agent workloads show that Pancake substantially outperforms existing frameworks, achieving more than 4.29x end-to-end throughput improvement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21477 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Pancake: Hierarchical Memory System for Multi-Agent LLM Serving Hu, Zhengding Pan, Zaifeng Kaur, Prabhleen Murthy, Vibha Yu, Zhongkai Guan, Yue Wang, Zhen Swanson, Steven Ding, Yufei Multiagent Systems In this work, we identify and address the core challenges of agentic memory management in LLM serving, where large-scale storage, frequent updates, and multiple coexisting agents jointly introduce complex and high-cost approximate nearest neighbor (ANN) searching problems. We present Pancake, a multi-tier agentic memory system that unifies three key techniques: (i) multi-level index caching for single agents, (ii) coordinated index management across multiple agents, and (iii) collaborative GPU-CPU acceleration. Pancake exposes easy-to-use interface that can be integrated into memory-based agents like Mem-GPT, and is compatible with agentic frameworks such as LangChain and LlamaIndex. Experiments on realistic agent workloads show that Pancake substantially outperforms existing frameworks, achieving more than 4.29x end-to-end throughput improvement. |
| title | Pancake: Hierarchical Memory System for Multi-Agent LLM Serving |
| topic | Multiagent Systems |
| url | https://arxiv.org/abs/2602.21477 |