Larimar: Large Language Models with Episodic Memory Control
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914920064352256 |
|---|---|
| author | Das, Payel Chaudhury, Subhajit Nelson, Elliot Melnyk, Igor Swaminathan, Sarath Dai, Sihui Lozano, Aurélie Kollias, Georgios Chenthamarakshan, Vijil Jiří Navrátil Dan, Soham Chen, Pin-Yu |
| author_facet | Das, Payel Chaudhury, Subhajit Nelson, Elliot Melnyk, Igor Swaminathan, Sarath Dai, Sihui Lozano, Aurélie Kollias, Georgios Chenthamarakshan, Vijil Jiří Navrátil Dan, Soham Chen, Pin-Yu |
| contents | Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory. Larimar's memory allows for dynamic, one-shot updates of knowledge without the need for computationally expensive re-training or fine-tuning. Experimental results on multiple fact editing benchmarks demonstrate that Larimar attains accuracy comparable to most competitive baselines, even in the challenging sequential editing setup, but also excels in speed - yielding speed-ups of 8-10x depending on the base LLM - as well as flexibility due to the proposed architecture being simple, LLM-agnostic, and hence general. We further provide mechanisms for selective fact forgetting, information leakage prevention, and input context length generalization with Larimar and show their effectiveness. Our code is available at https://github.com/IBM/larimar |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_11901 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Larimar: Large Language Models with Episodic Memory Control Das, Payel Chaudhury, Subhajit Nelson, Elliot Melnyk, Igor Swaminathan, Sarath Dai, Sihui Lozano, Aurélie Kollias, Georgios Chenthamarakshan, Vijil Jiří Navrátil Dan, Soham Chen, Pin-Yu Machine Learning Artificial Intelligence Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory. Larimar's memory allows for dynamic, one-shot updates of knowledge without the need for computationally expensive re-training or fine-tuning. Experimental results on multiple fact editing benchmarks demonstrate that Larimar attains accuracy comparable to most competitive baselines, even in the challenging sequential editing setup, but also excels in speed - yielding speed-ups of 8-10x depending on the base LLM - as well as flexibility due to the proposed architecture being simple, LLM-agnostic, and hence general. We further provide mechanisms for selective fact forgetting, information leakage prevention, and input context length generalization with Larimar and show their effectiveness. Our code is available at https://github.com/IBM/larimar |
| title | Larimar: Large Language Models with Episodic Memory Control |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2403.11901 |