Motion Graph Unleashed: A Novel Approach to Video Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909370754793472 |
|---|---|
| author | Zhong, Yiqi Liang, Luming Tang, Bohan Zharkov, Ilya Neumann, Ulrich |
| author_facet | Zhong, Yiqi Liang, Luming Tang, Bohan Zharkov, Ilya Neumann, Ulrich |
| contents | We introduce motion graph, a novel approach to the video prediction problem, which predicts future video frames from limited past data. The motion graph transforms patches of video frames into interconnected graph nodes, to comprehensively describe the spatial-temporal relationships among them. This representation overcomes the limitations of existing motion representations such as image differences, optical flow, and motion matrix that either fall short in capturing complex motion patterns or suffer from excessive memory consumption. We further present a video prediction pipeline empowered by motion graph, exhibiting substantial performance improvements and cost reductions. Experiments on various datasets, including UCF Sports, KITTI and Cityscapes, highlight the strong representative ability of motion graph. Especially on UCF Sports, our method matches and outperforms the SOTA methods with a significant reduction in model size by 78% and a substantial decrease in GPU memory utilization by 47%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22288 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Motion Graph Unleashed: A Novel Approach to Video Prediction Zhong, Yiqi Liang, Luming Tang, Bohan Zharkov, Ilya Neumann, Ulrich Computer Vision and Pattern Recognition We introduce motion graph, a novel approach to the video prediction problem, which predicts future video frames from limited past data. The motion graph transforms patches of video frames into interconnected graph nodes, to comprehensively describe the spatial-temporal relationships among them. This representation overcomes the limitations of existing motion representations such as image differences, optical flow, and motion matrix that either fall short in capturing complex motion patterns or suffer from excessive memory consumption. We further present a video prediction pipeline empowered by motion graph, exhibiting substantial performance improvements and cost reductions. Experiments on various datasets, including UCF Sports, KITTI and Cityscapes, highlight the strong representative ability of motion graph. Especially on UCF Sports, our method matches and outperforms the SOTA methods with a significant reduction in model size by 78% and a substantial decrease in GPU memory utilization by 47%. |
| title | Motion Graph Unleashed: A Novel Approach to Video Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.22288 |