Geometry-Aware Implicit Memory for Video World Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911741980442624 |
|---|---|
| author | Wei, Zhengxuan Guo, Xu Li, Xinghui Xiang, Xunzhi Wei, Min Zhu, Yiran Wang, Qiulin Wang, Xintao Wan, Pengfei Hou, Xiangwang Fan, Qi |
| author_facet | Wei, Zhengxuan Guo, Xu Li, Xinghui Xiang, Xunzhi Wei, Min Zhu, Yiran Wang, Qiulin Wang, Xintao Wan, Pengfei Hou, Xiangwang Fan, Qi |
| contents | Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_02436 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Geometry-Aware Implicit Memory for Video World Models Wei, Zhengxuan Guo, Xu Li, Xinghui Xiang, Xunzhi Wei, Min Zhu, Yiran Wang, Qiulin Wang, Xintao Wan, Pengfei Hou, Xiangwang Fan, Qi Computer Vision and Pattern Recognition Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines. |
| title | Geometry-Aware Implicit Memory for Video World Models |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2606.02436 |