OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910482641715200 |
|---|---|
| author | Sirko-Galouchenko, Sophia Boulch, Alexandre Gidaris, Spyros Bursuc, Andrei Vobecky, Antonin Pérez, Patrick Marlet, Renaud |
| author_facet | Sirko-Galouchenko, Sophia Boulch, Alexandre Gidaris, Spyros Bursuc, Andrei Vobecky, Antonin Pérez, Patrick Marlet, Renaud |
| contents | We introduce a self-supervised pretraining method, called OccFeat, for camera-only Bird's-Eye-View (BEV) segmentation networks. With OccFeat, we pretrain a BEV network via occupancy prediction and feature distillation tasks. Occupancy prediction provides a 3D geometric understanding of the scene to the model. However, the geometry learned is class-agnostic. Hence, we add semantic information to the model in the 3D space through distillation from a self-supervised pretrained image foundation model. Models pretrained with our method exhibit improved BEV semantic segmentation performance, particularly in low-data scenarios. Moreover, empirical results affirm the efficacy of integrating feature distillation with 3D occupancy prediction in our pretraining approach. Repository: https://github.com/valeoai/Occfeat |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_14027 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks Sirko-Galouchenko, Sophia Boulch, Alexandre Gidaris, Spyros Bursuc, Andrei Vobecky, Antonin Pérez, Patrick Marlet, Renaud Computer Vision and Pattern Recognition Machine Learning We introduce a self-supervised pretraining method, called OccFeat, for camera-only Bird's-Eye-View (BEV) segmentation networks. With OccFeat, we pretrain a BEV network via occupancy prediction and feature distillation tasks. Occupancy prediction provides a 3D geometric understanding of the scene to the model. However, the geometry learned is class-agnostic. Hence, we add semantic information to the model in the 3D space through distillation from a self-supervised pretrained image foundation model. Models pretrained with our method exhibit improved BEV semantic segmentation performance, particularly in low-data scenarios. Moreover, empirical results affirm the efficacy of integrating feature distillation with 3D occupancy prediction in our pretraining approach. Repository: https://github.com/valeoai/Occfeat |
| title | OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2404.14027 |