Long Context Modeling with Ranked Memory-Augmented Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917506754543616 |
|---|---|
| author | Alselwi, Ghadir Xue, Hao Jameel, Shoaib Suleiman, Basem Salim, Flora D. Razzak, Imran |
| author_facet | Alselwi, Ghadir Xue, Hao Jameel, Shoaib Suleiman, Basem Salim, Flora D. Razzak, Imran |
| contents | Effective long-term memory management is crucial for language models handling extended contexts. We introduce the Enhanced Ranked Memory Augmented Retrieval (ERMAR) framework, which dynamically ranks memory entries based on relevance. Unlike prior models, ERMAR employs a novel relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings, inspired by learning-to-rank techniques in information retrieval. By integrating historical usage patterns and adaptive retrieval, ERMAR achieves state-of-the-art results on standard benchmarks, demonstrating superior scalability and performance in long-context tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14800 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Long Context Modeling with Ranked Memory-Augmented Retrieval Alselwi, Ghadir Xue, Hao Jameel, Shoaib Suleiman, Basem Salim, Flora D. Razzak, Imran Information Retrieval Artificial Intelligence Machine Learning Effective long-term memory management is crucial for language models handling extended contexts. We introduce the Enhanced Ranked Memory Augmented Retrieval (ERMAR) framework, which dynamically ranks memory entries based on relevance. Unlike prior models, ERMAR employs a novel relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings, inspired by learning-to-rank techniques in information retrieval. By integrating historical usage patterns and adaptive retrieval, ERMAR achieves state-of-the-art results on standard benchmarks, demonstrating superior scalability and performance in long-context tasks. |
| title | Long Context Modeling with Ranked Memory-Augmented Retrieval |
| topic | Information Retrieval Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2503.14800 |