Go Beyond Earth: Understanding Human Actions and Scenes in Microgravity Environments
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908903664517120 |
|---|---|
| author | Wen, Di Qi, Lei Peng, Kunyu Yang, Kailun Teng, Fei Luo, Ao Fu, Jia Chen, Yufan Liu, Ruiping Shi, Yitian Sarfraz, M. Saquib Stiefelhagen, Rainer |
| author_facet | Wen, Di Qi, Lei Peng, Kunyu Yang, Kailun Teng, Fei Luo, Ao Fu, Jia Chen, Yufan Liu, Ruiping Shi, Yitian Sarfraz, M. Saquib Stiefelhagen, Rainer |
| contents | Despite substantial progress in video understanding, most existing datasets are limited to Earth's gravitational conditions. However, microgravity alters human motion, interactions, and visual semantics, revealing a critical gap for real-world vision systems. This presents a challenge for domain-robust video understanding in safety-critical space applications. To address this, we introduce MicroG-4M, the first benchmark for spatio-temporal and semantic understanding of human activities in microgravity. Constructed from real-world space missions and cinematic simulations, the dataset includes 4,759 clips covering 50 actions, 1,238 context-rich captions, and over 7,000 question-answer pairs on astronaut activities and scene understanding. MicroG-4M supports three core tasks: fine-grained multi-label action recognition, temporal video captioning, and visual question answering, enabling a comprehensive evaluation of both spatial localization and semantic reasoning in microgravity contexts. We establish baselines using state-of-the-art models. All data, annotations, and code are available at https://github.com/LEI-QI-233/HAR-in-Space. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02845 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Go Beyond Earth: Understanding Human Actions and Scenes in Microgravity Environments Wen, Di Qi, Lei Peng, Kunyu Yang, Kailun Teng, Fei Luo, Ao Fu, Jia Chen, Yufan Liu, Ruiping Shi, Yitian Sarfraz, M. Saquib Stiefelhagen, Rainer Computer Vision and Pattern Recognition Despite substantial progress in video understanding, most existing datasets are limited to Earth's gravitational conditions. However, microgravity alters human motion, interactions, and visual semantics, revealing a critical gap for real-world vision systems. This presents a challenge for domain-robust video understanding in safety-critical space applications. To address this, we introduce MicroG-4M, the first benchmark for spatio-temporal and semantic understanding of human activities in microgravity. Constructed from real-world space missions and cinematic simulations, the dataset includes 4,759 clips covering 50 actions, 1,238 context-rich captions, and over 7,000 question-answer pairs on astronaut activities and scene understanding. MicroG-4M supports three core tasks: fine-grained multi-label action recognition, temporal video captioning, and visual question answering, enabling a comprehensive evaluation of both spatial localization and semantic reasoning in microgravity contexts. We establish baselines using state-of-the-art models. All data, annotations, and code are available at https://github.com/LEI-QI-233/HAR-in-Space. |
| title | Go Beyond Earth: Understanding Human Actions and Scenes in Microgravity Environments |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2506.02845 |