Lifelong Learning of Video Diffusion Models From a Single Video Stream
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912458488152064 |
|---|---|
| author | Yoo, Jason He, Yingchen Naderiparizi, Saeid Green, Dylan van de Ven, Gido M. Pleiss, Geoff Wood, Frank |
| author_facet | Yoo, Jason He, Yingchen Naderiparizi, Saeid Green, Dylan van de Ven, Gido M. Pleiss, Geoff Wood, Frank |
| contents | This work demonstrates that training autoregressive video diffusion models from a single video stream$\unicode{x2013}$resembling the experience of embodied agents$\unicode{x2013}$is not only possible, but can also be as effective as standard offline training given the same number of gradient steps. Our work further reveals that this main result can be achieved using experience replay methods that only retain a subset of the preceding video stream. To support training and evaluation in this setting, we introduce four new datasets for streaming lifelong generative video modeling: Lifelong Bouncing Balls, Lifelong 3D Maze, Lifelong Drive, and Lifelong PLAICraft, each consisting of one million consecutive frames from environments of increasing complexity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04814 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Lifelong Learning of Video Diffusion Models From a Single Video Stream Yoo, Jason He, Yingchen Naderiparizi, Saeid Green, Dylan van de Ven, Gido M. Pleiss, Geoff Wood, Frank Computer Vision and Pattern Recognition Machine Learning This work demonstrates that training autoregressive video diffusion models from a single video stream$\unicode{x2013}$resembling the experience of embodied agents$\unicode{x2013}$is not only possible, but can also be as effective as standard offline training given the same number of gradient steps. Our work further reveals that this main result can be achieved using experience replay methods that only retain a subset of the preceding video stream. To support training and evaluation in this setting, we introduce four new datasets for streaming lifelong generative video modeling: Lifelong Bouncing Balls, Lifelong 3D Maze, Lifelong Drive, and Lifelong PLAICraft, each consisting of one million consecutive frames from environments of increasing complexity. |
| title | Lifelong Learning of Video Diffusion Models From a Single Video Stream |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2406.04814 |