Depth Matters: Multimodal RGB-D Perception for Robust Autonomous Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909900023529472 |
|---|---|
| author | Clement, Mihaela-Larisa Farsang, Mónika Resch, Felix Stanusoiu, Mihai-Teodor Grosu, Radu |
| author_facet | Clement, Mihaela-Larisa Farsang, Mónika Resch, Felix Stanusoiu, Mihai-Teodor Grosu, Radu |
| contents | Autonomous agents that rely purely on perception to make real-time control decisions require efficient and robust architectures. In this work, we demonstrate that augmenting RGB input with depth information significantly enhances our agents' ability to predict steering commands compared to using RGB alone. We benchmark lightweight recurrent controllers that leverage the fused RGB-D features for sequential decision-making. To train our models, we collect high-quality data using a small-scale autonomous car controlled by an expert driver via a physical steering wheel, capturing varying levels of steering difficulty. Our models were successfully deployed on real hardware and inherently avoided dynamic and static obstacles, under out-of-distribution conditions. Specifically, our findings reveal that the early fusion of depth data results in a highly robust controller, which remains effective even with frame drops and increased noise levels, without compromising the network's focus on the task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16711 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Depth Matters: Multimodal RGB-D Perception for Robust Autonomous Agents Clement, Mihaela-Larisa Farsang, Mónika Resch, Felix Stanusoiu, Mihai-Teodor Grosu, Radu Robotics Computer Vision and Pattern Recognition Machine Learning Autonomous agents that rely purely on perception to make real-time control decisions require efficient and robust architectures. In this work, we demonstrate that augmenting RGB input with depth information significantly enhances our agents' ability to predict steering commands compared to using RGB alone. We benchmark lightweight recurrent controllers that leverage the fused RGB-D features for sequential decision-making. To train our models, we collect high-quality data using a small-scale autonomous car controlled by an expert driver via a physical steering wheel, capturing varying levels of steering difficulty. Our models were successfully deployed on real hardware and inherently avoided dynamic and static obstacles, under out-of-distribution conditions. Specifically, our findings reveal that the early fusion of depth data results in a highly robust controller, which remains effective even with frame drops and increased noise levels, without compromising the network's focus on the task. |
| title | Depth Matters: Multimodal RGB-D Perception for Robust Autonomous Agents |
| topic | Robotics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2503.16711 |