MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913749154136064 |
|---|---|
| author | Zatsarynna, Olga Bahrami, Emad Farha, Yazan Abu Francesca, Gianpiero Gall, Juergen |
| author_facet | Zatsarynna, Olga Bahrami, Emad Farha, Yazan Abu Francesca, Gianpiero Gall, Juergen |
| contents | Long-term dense action anticipation is very challenging since it requires predicting actions and their durations several minutes into the future based on provided video observations. To model the uncertainty of future outcomes, stochastic models predict several potential future action sequences for the same observation. Recent work has further proposed to incorporate uncertainty modelling for observed frames by simultaneously predicting per-frame past and future actions in a unified manner. While such joint modelling of actions is beneficial, it requires long-range temporal capabilities to connect events across distant past and future time points. However, the previous work struggles to achieve such a long-range understanding due to its limited and/or sparse receptive field. To alleviate this issue, we propose a novel MANTA (MAmba for ANTicipation) network. Our model enables effective long-term temporal modelling even for very long sequences while maintaining linear complexity in sequence length. We demonstrate that our approach achieves state-of-the-art results on three datasets - Breakfast, 50Salads, and Assembly101 - while also significantly improving computational and memory efficiency. Our code is available at https://github.com/olga-zats/DIFF_MANTA . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_08837 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipation Zatsarynna, Olga Bahrami, Emad Farha, Yazan Abu Francesca, Gianpiero Gall, Juergen Computer Vision and Pattern Recognition Long-term dense action anticipation is very challenging since it requires predicting actions and their durations several minutes into the future based on provided video observations. To model the uncertainty of future outcomes, stochastic models predict several potential future action sequences for the same observation. Recent work has further proposed to incorporate uncertainty modelling for observed frames by simultaneously predicting per-frame past and future actions in a unified manner. While such joint modelling of actions is beneficial, it requires long-range temporal capabilities to connect events across distant past and future time points. However, the previous work struggles to achieve such a long-range understanding due to its limited and/or sparse receptive field. To alleviate this issue, we propose a novel MANTA (MAmba for ANTicipation) network. Our model enables effective long-term temporal modelling even for very long sequences while maintaining linear complexity in sequence length. We demonstrate that our approach achieves state-of-the-art results on three datasets - Breakfast, 50Salads, and Assembly101 - while also significantly improving computational and memory efficiency. Our code is available at https://github.com/olga-zats/DIFF_MANTA . |
| title | MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.08837 |