Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.13352 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908775292600320 |
|---|---|
| author | Lu, Yuxing Tamo, J. Ben Zhao, Weichen Sun, Nan Zhong, Yishan Shi, Wenqi Wang, Jinzhuo Wang, May D. |
| author_facet | Lu, Yuxing Tamo, J. Ben Zhao, Weichen Sun, Nan Zhong, Yishan Shi, Wenqi Wang, Jinzhuo Wang, May D. |
| contents | Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_13352 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction Lu, Yuxing Tamo, J. Ben Zhao, Weichen Sun, Nan Zhong, Yishan Shi, Wenqi Wang, Jinzhuo Wang, May D. Computation and Language Artificial Intelligence Multiagent Systems Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation. |
| title | LLM-as-RNN: A Recurrent Language Model for Memory Updates and Sequence Prediction |
| topic | Computation and Language Artificial Intelligence Multiagent Systems |
| url | https://arxiv.org/abs/2601.13352 |