Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | English |
| Published: |
Zenodo
2026
|
| Online Access: | https://doi.org/10.5281/zenodo.18436987 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866902175893946368 |
|---|---|
| author | Revista, Zen |
| author_facet | Revista, Zen |
| contents | <p><strong>Large Language Models (LLMs)</strong> have revolutionized natural language processing, but they face a critical limitation: the <strong>context window</strong>. Despite recent advances that have expanded context windows to millions of tokens, models still suffer from <strong>“context rot”</strong> — a systematic degradation in performance as input length increases.</p> <p>This paper explores <strong>Recursive Language Models (RLMs)</strong>, a novel inference paradigm inspired by <strong>level-of-detail (LOD)</strong> and <strong>streaming techniques</strong> from open-world video games such as <em>Minecraft</em>, <em>Grand Theft Auto</em>, <em>Assassin’s Creed</em>, <em>Red Dead Redemption</em>, and <em>Cyberpunk 2077</em>.</p> <p>RLMs conceptualize large prompts as <strong>external environments</strong> that models can <strong>programmatically explore through recursive sub-queries</strong>, enabling effective processing of inputs up to <strong>100× beyond base model context windows</strong>.</p> <p>We examine the <strong>technical parallels</strong> between game rendering optimization and context management, analyze <strong>empirical results</strong> showing that RLMs outperform vanilla frontier LLMs by up to <strong>114% on information-dense tasks</strong>, and discuss the <strong>implications</strong> for future AI systems capable of handling <strong>arbitrarily long contexts</strong>.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18436987 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Recursive Language Models: Borrowing from Video Game Rendering to Overcome Context Window Limitations in Large Language Models Revista, Zen <p><strong>Large Language Models (LLMs)</strong> have revolutionized natural language processing, but they face a critical limitation: the <strong>context window</strong>. Despite recent advances that have expanded context windows to millions of tokens, models still suffer from <strong>“context rot”</strong> — a systematic degradation in performance as input length increases.</p> <p>This paper explores <strong>Recursive Language Models (RLMs)</strong>, a novel inference paradigm inspired by <strong>level-of-detail (LOD)</strong> and <strong>streaming techniques</strong> from open-world video games such as <em>Minecraft</em>, <em>Grand Theft Auto</em>, <em>Assassin’s Creed</em>, <em>Red Dead Redemption</em>, and <em>Cyberpunk 2077</em>.</p> <p>RLMs conceptualize large prompts as <strong>external environments</strong> that models can <strong>programmatically explore through recursive sub-queries</strong>, enabling effective processing of inputs up to <strong>100× beyond base model context windows</strong>.</p> <p>We examine the <strong>technical parallels</strong> between game rendering optimization and context management, analyze <strong>empirical results</strong> showing that RLMs outperform vanilla frontier LLMs by up to <strong>114% on information-dense tasks</strong>, and discuss the <strong>implications</strong> for future AI systems capable of handling <strong>arbitrarily long contexts</strong>.</p> |
| title | Recursive Language Models: Borrowing from Video Game Rendering to Overcome Context Window Limitations in Large Language Models |
| url | https://doi.org/10.5281/zenodo.18436987 |