MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908480579829760 |
|---|---|
| author | Xiao, Lixing Lu, Shunlin Pi, Huaijin Fan, Ke Pan, Liang Zhou, Yueer Feng, Ziyong Zhou, Xiaowei Peng, Sida Wang, Jingbo |
| author_facet | Xiao, Lixing Lu, Shunlin Pi, Huaijin Fan, Ke Pan, Liang Zhou, Yueer Feng, Ziyong Zhou, Xiaowei Peng, Sida Wang, Jingbo |
| contents | This paper addresses the challenge of text-conditioned streaming motion generation, which requires us to predict the next-step human pose based on variable-length historical motions and incoming texts. Existing methods struggle to achieve streaming motion generation, e.g., diffusion models are constrained by pre-defined motion lengths, while GPT-based methods suffer from delayed response and error accumulation problem due to discretized non-causal tokenization. To solve these problems, we propose MotionStreamer, a novel framework that incorporates a continuous causal latent space into a probabilistic autoregressive model. The continuous latents mitigate information loss caused by discretization and effectively reduce error accumulation during long-term autoregressive generation. In addition, by establishing temporal causal dependencies between current and historical motion latents, our model fully utilizes the available information to achieve accurate online motion decoding. Experiments show that our method outperforms existing approaches while offering more applications, including multi-round generation, long-term generation, and dynamic motion composition. Project Page: https://zju3dv.github.io/MotionStreamer/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_15451 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space Xiao, Lixing Lu, Shunlin Pi, Huaijin Fan, Ke Pan, Liang Zhou, Yueer Feng, Ziyong Zhou, Xiaowei Peng, Sida Wang, Jingbo Computer Vision and Pattern Recognition This paper addresses the challenge of text-conditioned streaming motion generation, which requires us to predict the next-step human pose based on variable-length historical motions and incoming texts. Existing methods struggle to achieve streaming motion generation, e.g., diffusion models are constrained by pre-defined motion lengths, while GPT-based methods suffer from delayed response and error accumulation problem due to discretized non-causal tokenization. To solve these problems, we propose MotionStreamer, a novel framework that incorporates a continuous causal latent space into a probabilistic autoregressive model. The continuous latents mitigate information loss caused by discretization and effectively reduce error accumulation during long-term autoregressive generation. In addition, by establishing temporal causal dependencies between current and historical motion latents, our model fully utilizes the available information to achieve accurate online motion decoding. Experiments show that our method outperforms existing approaches while offering more applications, including multi-round generation, long-term generation, and dynamic motion composition. Project Page: https://zju3dv.github.io/MotionStreamer/ |
| title | MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.15451 |