Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2512.17012 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914465541259264 |
|---|---|
| author | Yang, Chiao-An Hachiuma, Ryo Liu, Sifei Radhakrishnan, Subhashree Yeh, Raymond A. Wang, Yu-Chiang Frank Chen, Min-Hung |
| author_facet | Yang, Chiao-An Hachiuma, Ryo Liu, Sifei Radhakrishnan, Subhashree Yeh, Raymond A. Wang, Yu-Chiang Frank Chen, Min-Hung |
| contents | Despite advances in Multimodal LLMs (MLLMs), their ability to reason over 3D structures and temporal dynamics remains limited, constrained by weak 4D perception and temporal understanding. Existing 3D and 4D Video Question Answering (VQA) benchmarks also emphasize static scenes and lack region-level prompting. We tackle these issues by introducing: (a) 4D-RGPT, a specialized MLLM designed to capture 4D representations from video inputs with enhanced temporal perception; (b) Perceptual 4D Distillation (P4D), a training framework that transfers 4D representations from a frozen expert model into 4D-RGPT for comprehensive 4D perception; and (c) R4D-Bench, a benchmark for depth-aware dynamic scenes with region-level prompting, built via a hybrid automated and human-verified pipeline. Our 4D-RGPT achieves notable improvements on both existing 4D VQA benchmarks and the proposed R4D-Bench benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_17012 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | 4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation Yang, Chiao-An Hachiuma, Ryo Liu, Sifei Radhakrishnan, Subhashree Yeh, Raymond A. Wang, Yu-Chiang Frank Chen, Min-Hung Computer Vision and Pattern Recognition Despite advances in Multimodal LLMs (MLLMs), their ability to reason over 3D structures and temporal dynamics remains limited, constrained by weak 4D perception and temporal understanding. Existing 3D and 4D Video Question Answering (VQA) benchmarks also emphasize static scenes and lack region-level prompting. We tackle these issues by introducing: (a) 4D-RGPT, a specialized MLLM designed to capture 4D representations from video inputs with enhanced temporal perception; (b) Perceptual 4D Distillation (P4D), a training framework that transfers 4D representations from a frozen expert model into 4D-RGPT for comprehensive 4D perception; and (c) R4D-Bench, a benchmark for depth-aware dynamic scenes with region-level prompting, built via a hybrid automated and human-verified pipeline. Our 4D-RGPT achieves notable improvements on both existing 4D VQA benchmarks and the proposed R4D-Bench benchmark. |
| title | 4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.17012 |