LightMem: Lightweight and Efficient Memory-Augmented Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914360632279040 |
|---|---|
| author | Fang, Jizhan Deng, Xinle Xu, Haoming Jiang, Ziyan Tang, Yuqi Xu, Ziwen Deng, Shumin Yao, Yunzhi Wang, Mengru Qiao, Shuofei Chen, Huajun Zhang, Ningyu |
| author_facet | Fang, Jizhan Deng, Xinle Xu, Haoming Jiang, Ziyan Tang, Yuqi Xu, Ziwen Deng, Shumin Yao, Yunzhi Wang, Mengru Qiao, Shuofei Chen, Huajun Zhang, Ningyu |
| contents | Despite their remarkable capabilities, Large Language Models (LLMs) struggle to effectively leverage historical interaction information in dynamic and complex environments. Memory systems enable LLMs to move beyond stateless interactions by introducing persistent information storage, retrieval, and utilization mechanisms. However, existing memory systems often introduce substantial time and computational overhead. To this end, we introduce a new memory system called LightMem, which strikes a balance between the performance and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of human memory, LightMem organizes memory into three complementary stages. First, cognition-inspired sensory memory rapidly filters irrelevant information through lightweight compression and groups information according to their topics. Next, topic-aware short-term memory consolidates these topic-based groups, organizing and summarizing content for more structured access. Finally, long-term memory with sleep-time update employs an offline procedure that decouples consolidation from online inference. On LongMemEval and LoCoMo, using GPT and Qwen backbones, LightMem consistently surpasses strong baselines, improving QA accuracy by up to 7.7% / 29.3%, reducing total token usage by up to 38x / 20.9x and API calls by up to 30x / 55.5x, while purely online test-time costs are even lower, achieving up to 106x / 117x token reduction and 159x / 310x fewer API calls. The code is available at https://github.com/zjunlp/LightMem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_18866 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LightMem: Lightweight and Efficient Memory-Augmented Generation Fang, Jizhan Deng, Xinle Xu, Haoming Jiang, Ziyan Tang, Yuqi Xu, Ziwen Deng, Shumin Yao, Yunzhi Wang, Mengru Qiao, Shuofei Chen, Huajun Zhang, Ningyu Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multiagent Systems Despite their remarkable capabilities, Large Language Models (LLMs) struggle to effectively leverage historical interaction information in dynamic and complex environments. Memory systems enable LLMs to move beyond stateless interactions by introducing persistent information storage, retrieval, and utilization mechanisms. However, existing memory systems often introduce substantial time and computational overhead. To this end, we introduce a new memory system called LightMem, which strikes a balance between the performance and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of human memory, LightMem organizes memory into three complementary stages. First, cognition-inspired sensory memory rapidly filters irrelevant information through lightweight compression and groups information according to their topics. Next, topic-aware short-term memory consolidates these topic-based groups, organizing and summarizing content for more structured access. Finally, long-term memory with sleep-time update employs an offline procedure that decouples consolidation from online inference. On LongMemEval and LoCoMo, using GPT and Qwen backbones, LightMem consistently surpasses strong baselines, improving QA accuracy by up to 7.7% / 29.3%, reducing total token usage by up to 38x / 20.9x and API calls by up to 30x / 55.5x, while purely online test-time costs are even lower, achieving up to 106x / 117x token reduction and 159x / 310x fewer API calls. The code is available at https://github.com/zjunlp/LightMem. |
| title | LightMem: Lightweight and Efficient Memory-Augmented Generation |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2510.18866 |