LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911865600212992 |
|---|---|
| author | Hung, Wei-Chih Casser, Vincent Kretzschmar, Henrik Hwang, Jyh-Jing Anguelov, Dragomir |
| author_facet | Hung, Wei-Chih Casser, Vincent Kretzschmar, Henrik Hwang, Jyh-Jing Anguelov, Dragomir |
| contents | The 3D Average Precision (3D AP) relies on the intersection over union between predictions and ground truth objects. However, camera-only detectors have limited depth accuracy, which may cause otherwise reasonable predictions that suffer from such longitudinal localization errors to be treated as false positives. We therefore propose variants of the 3D AP metric to be more permissive with respect to depth estimation errors. Specifically, our novel longitudinal error tolerant metrics, LET-3D-AP and LET-3D-APL, allow longitudinal localization errors of the prediction boxes up to a given tolerance. To evaluate the proposed metrics, we also construct a new test set for the Waymo Open Dataset, tailored to camera-only 3D detection methods. Surprisingly, we find that state-of-the-art camera-based detectors can outperform popular LiDAR-based detectors with our new metrics past at 10% depth error tolerance, suggesting that existing camera-based detectors already have the potential to surpass LiDAR-based detectors in downstream applications. We believe the proposed metrics and the new benchmark dataset will facilitate advances in the field of camera-only 3D detection by providing more informative signals that can better indicate the system-level performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2206_07705 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection Hung, Wei-Chih Casser, Vincent Kretzschmar, Henrik Hwang, Jyh-Jing Anguelov, Dragomir Computer Vision and Pattern Recognition The 3D Average Precision (3D AP) relies on the intersection over union between predictions and ground truth objects. However, camera-only detectors have limited depth accuracy, which may cause otherwise reasonable predictions that suffer from such longitudinal localization errors to be treated as false positives. We therefore propose variants of the 3D AP metric to be more permissive with respect to depth estimation errors. Specifically, our novel longitudinal error tolerant metrics, LET-3D-AP and LET-3D-APL, allow longitudinal localization errors of the prediction boxes up to a given tolerance. To evaluate the proposed metrics, we also construct a new test set for the Waymo Open Dataset, tailored to camera-only 3D detection methods. Surprisingly, we find that state-of-the-art camera-based detectors can outperform popular LiDAR-based detectors with our new metrics past at 10% depth error tolerance, suggesting that existing camera-based detectors already have the potential to surpass LiDAR-based detectors in downstream applications. We believe the proposed metrics and the new benchmark dataset will facilitate advances in the field of camera-only 3D detection by providing more informative signals that can better indicate the system-level performance. |
| title | LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2206.07705 |