Pooling Engram Conditional Memory in Large Language Models using CXL
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910048456802304 |
|---|---|
| author | Ma, Ruiyang Ma, Teng Su, Zhiyuan Zha, Hantian Zhao, Xinpeng Shang, Xuchun Yi, Xingrui Liu, Zheng Cao, Zhu Wu, An Dou, Zhichong Liu, Ziqian Kuang, Daikang Luo, Guojie |
| author_facet | Ma, Ruiyang Ma, Teng Su, Zhiyuan Zha, Hantian Zhao, Xinpeng Shang, Xuchun Yi, Xingrui Liu, Zheng Cao, Zhu Wu, An Dou, Zhichong Liu, Ziqian Kuang, Daikang Luo, Guojie |
| contents | Engram conditional memory has emerged as a promising component for LLMs by decoupling static knowledge lookup from dynamic computation. Since Engram exhibits sparse access patterns and supports prefetching, its massive embedding tables are well-suited for offloading to lower-tier memory. In this paper, we propose using Compute Express Link (CXL) memory pool for Engram storage. Compared to RDMA, CXL provides fine-grained and low-latency access required by minimal and discrete retrieval patterns of Engram. We integrate the CXL-based Engram pool into SGLang, achieving near-DRAM end-to-end performance. This provides a scalable and cost-efficient storage solution for future Engram-integrated LLMs without compromising inference performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10087 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Pooling Engram Conditional Memory in Large Language Models using CXL Ma, Ruiyang Ma, Teng Su, Zhiyuan Zha, Hantian Zhao, Xinpeng Shang, Xuchun Yi, Xingrui Liu, Zheng Cao, Zhu Wu, An Dou, Zhichong Liu, Ziqian Kuang, Daikang Luo, Guojie Hardware Architecture Distributed, Parallel, and Cluster Computing Engram conditional memory has emerged as a promising component for LLMs by decoupling static knowledge lookup from dynamic computation. Since Engram exhibits sparse access patterns and supports prefetching, its massive embedding tables are well-suited for offloading to lower-tier memory. In this paper, we propose using Compute Express Link (CXL) memory pool for Engram storage. Compared to RDMA, CXL provides fine-grained and low-latency access required by minimal and discrete retrieval patterns of Engram. We integrate the CXL-based Engram pool into SGLang, achieving near-DRAM end-to-end performance. This provides a scalable and cost-efficient storage solution for future Engram-integrated LLMs without compromising inference performance. |
| title | Pooling Engram Conditional Memory in Large Language Models using CXL |
| topic | Hardware Architecture Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2603.10087 |