Story-Iter: A Training-free Iterative Paradigm for Long Story Visualization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912904619491328 |
|---|---|
| author | Mao, Jiawei Huang, Xiaoke Xie, Yunfei Chang, Yuanqi Hui, Mude Xu, Bingjie Zheng, Zeyu Wang, Zirui Xie, Cihang Zhou, Yuyin |
| author_facet | Mao, Jiawei Huang, Xiaoke Xie, Yunfei Chang, Yuanqi Hui, Mude Xu, Bingjie Zheng, Zeyu Wang, Zirui Xie, Cihang Zhou, Yuyin |
| contents | This paper introduces Story-Iter, a new training-free iterative paradigm to enhance long-story generation. Unlike existing methods that rely on fixed reference images to construct a complete story, our approach features a novel external iterative paradigm, extending beyond the internal iterative denoising steps of diffusion models, to continuously refine each generated image by incorporating all reference images from the previous round. To achieve this, we propose a plug-and-play, training-free global reference cross-attention (GRCA) module, modeling all reference frames with global embeddings, ensuring semantic consistency in long sequences. By progressively incorporating holistic visual context and text constraints, our iterative paradigm enables precise generation with fine-grained interactions, optimizing the story visualization step-by-step. Extensive experiments in the official story visualization dataset and our long story benchmark demonstrate that Story-Iter's state-of-the-art performance in long-story visualization (up to 100 frames) excels in both semantic consistency and fine-grained interactions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06244 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Story-Iter: A Training-free Iterative Paradigm for Long Story Visualization Mao, Jiawei Huang, Xiaoke Xie, Yunfei Chang, Yuanqi Hui, Mude Xu, Bingjie Zheng, Zeyu Wang, Zirui Xie, Cihang Zhou, Yuyin Computer Vision and Pattern Recognition This paper introduces Story-Iter, a new training-free iterative paradigm to enhance long-story generation. Unlike existing methods that rely on fixed reference images to construct a complete story, our approach features a novel external iterative paradigm, extending beyond the internal iterative denoising steps of diffusion models, to continuously refine each generated image by incorporating all reference images from the previous round. To achieve this, we propose a plug-and-play, training-free global reference cross-attention (GRCA) module, modeling all reference frames with global embeddings, ensuring semantic consistency in long sequences. By progressively incorporating holistic visual context and text constraints, our iterative paradigm enables precise generation with fine-grained interactions, optimizing the story visualization step-by-step. Extensive experiments in the official story visualization dataset and our long story benchmark demonstrate that Story-Iter's state-of-the-art performance in long-story visualization (up to 100 frames) excels in both semantic consistency and fine-grained interactions. |
| title | Story-Iter: A Training-free Iterative Paradigm for Long Story Visualization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.06244 |