A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911378800902144 |
|---|---|
| author | Wang, Yizhou Pusegaonkar, Sameer Wang, Yuxing Li, Anqi Kumar, Vishal Sethi, Chetan Aiyer, Ganapathy He, Yun Thakkar, Kartikay Rathi, Swapnil Rupde, Bhushan Tang, Zheng Biswas, Sujit |
| author_facet | Wang, Yizhou Pusegaonkar, Sameer Wang, Yuxing Li, Anqi Kumar, Vishal Sethi, Chetan Aiyer, Ganapathy He, Yun Thakkar, Kartikay Rathi, Swapnil Rupde, Bhushan Tang, Zheng Biswas, Sujit |
| contents | Accurate 3D object perception and multi-target multi-camera (MTMC) tracking are fundamental for the digital transformation of industrial infrastructure. However, transitioning "inside-out" autonomous driving models to "outside-in" static camera networks presents significant challenges due to heterogeneous camera placements and extreme occlusion. In this paper, we present an adapted Sparse4D framework specifically optimized for large-scale infrastructure environments. Our system leverages absolute world-coordinate geometric priors and introduces an occlusion-aware ReID embedding module to maintain identity stability across distributed sensor networks. To bridge the Sim2Real domain gap without manual labeling, we employ a generative data augmentation strategy using the NVIDIA COSMOS framework, creating diverse environmental styles that enhance the model's appearance-invariance. Evaluated on the AI City Challenge 2025 benchmark, our camera-only framework achieves a state-of-the-art HOTA of $45.22$. Furthermore, we address real-time deployment constraints by developing an optimized TensorRT plugin for Multi-Scale Deformable Aggregation (MSDA). Our hardware-accelerated implementation achieves a $2.15\times$ speedup on modern GPU architectures, enabling a single Blackwell-class GPU to support over 64 concurrent camera streams. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_10819 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems Wang, Yizhou Pusegaonkar, Sameer Wang, Yuxing Li, Anqi Kumar, Vishal Sethi, Chetan Aiyer, Ganapathy He, Yun Thakkar, Kartikay Rathi, Swapnil Rupde, Bhushan Tang, Zheng Biswas, Sujit Computer Vision and Pattern Recognition Accurate 3D object perception and multi-target multi-camera (MTMC) tracking are fundamental for the digital transformation of industrial infrastructure. However, transitioning "inside-out" autonomous driving models to "outside-in" static camera networks presents significant challenges due to heterogeneous camera placements and extreme occlusion. In this paper, we present an adapted Sparse4D framework specifically optimized for large-scale infrastructure environments. Our system leverages absolute world-coordinate geometric priors and introduces an occlusion-aware ReID embedding module to maintain identity stability across distributed sensor networks. To bridge the Sim2Real domain gap without manual labeling, we employ a generative data augmentation strategy using the NVIDIA COSMOS framework, creating diverse environmental styles that enhance the model's appearance-invariance. Evaluated on the AI City Challenge 2025 benchmark, our camera-only framework achieves a state-of-the-art HOTA of $45.22$. Furthermore, we address real-time deployment constraints by developing an optimized TensorRT plugin for Multi-Scale Deformable Aggregation (MSDA). Our hardware-accelerated implementation achieves a $2.15\times$ speedup on modern GPU architectures, enabling a single Blackwell-class GPU to support over 64 concurrent camera streams. |
| title | A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.10819 |