TTOM: Test-Time Optimization and Memorization for Compositional Video Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918362565574656 |
|---|---|
| author | Qu, Leigang Wang, Ziyang Zheng, Na Wang, Wenjie Nie, Liqiang Chua, Tat-Seng |
| author_facet | Qu, Leigang Wang, Ziyang Zheng, Na Wang, Wenjie Nie, Liqiang Chua, Tat-Seng |
| contents | Video Foundation Models (VFMs) exhibit remarkable visual generation performance, but struggle in compositional scenarios (e.g., motion, numeracy, and spatial relation). In this work, we introduce Test-Time Optimization and Memorization (TTOM), a training-free framework that aligns VFM outputs with spatiotemporal layouts during inference for better text-image alignment. Rather than direct intervention to latents or attention per-sample in existing work, we integrate and optimize new parameters guided by a general layout-attention objective. Furthermore, we formulate video generation within a streaming setting, and maintain historical optimization contexts with a parametric memory mechanism that supports flexible operations, such as insert, read, update, and delete. Notably, we found that TTOM disentangles compositional world knowledge, showing powerful transferability and generalization. Experimental results on the T2V-CompBench and Vbench benchmarks establish TTOM as an effective, practical, scalable, and efficient framework to achieve cross-modal alignment for compositional video generation on the fly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07940 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TTOM: Test-Time Optimization and Memorization for Compositional Video Generation Qu, Leigang Wang, Ziyang Zheng, Na Wang, Wenjie Nie, Liqiang Chua, Tat-Seng Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Multimedia Video Foundation Models (VFMs) exhibit remarkable visual generation performance, but struggle in compositional scenarios (e.g., motion, numeracy, and spatial relation). In this work, we introduce Test-Time Optimization and Memorization (TTOM), a training-free framework that aligns VFM outputs with spatiotemporal layouts during inference for better text-image alignment. Rather than direct intervention to latents or attention per-sample in existing work, we integrate and optimize new parameters guided by a general layout-attention objective. Furthermore, we formulate video generation within a streaming setting, and maintain historical optimization contexts with a parametric memory mechanism that supports flexible operations, such as insert, read, update, and delete. Notably, we found that TTOM disentangles compositional world knowledge, showing powerful transferability and generalization. Experimental results on the T2V-CompBench and Vbench benchmarks establish TTOM as an effective, practical, scalable, and efficient framework to achieve cross-modal alignment for compositional video generation on the fly. |
| title | TTOM: Test-Time Optimization and Memorization for Compositional Video Generation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Multimedia |
| url | https://arxiv.org/abs/2510.07940 |