ProtoDepth: Unsupervised Continual Depth Completion with Prototypes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912277467234304 |
|---|---|
| author | Rim, Patrick Park, Hyoungseob Gangopadhyay, S. Zeng, Ziyao Chung, Younjoon Wong, Alex |
| author_facet | Rim, Patrick Park, Hyoungseob Gangopadhyay, S. Zeng, Ziyao Chung, Younjoon Wong, Alex |
| contents | We present ProtoDepth, a novel prototype-based approach for continual learning of unsupervised depth completion, the multimodal 3D reconstruction task of predicting dense depth maps from RGB images and sparse point clouds. The unsupervised learning paradigm is well-suited for continual learning, as ground truth is not needed. However, when training on new non-stationary distributions, depth completion models will catastrophically forget previously learned information. We address forgetting by learning prototype sets that adapt the latent features of a frozen pretrained model to new domains. Since the original weights are not modified, ProtoDepth does not forget when test-time domain identity is known. To extend ProtoDepth to the challenging setting where the test-time domain identity is withheld, we propose to learn domain descriptors that enable the model to select the appropriate prototype set for inference. We evaluate ProtoDepth on benchmark dataset sequences, where we reduce forgetting compared to baselines by 52.2% for indoor and 53.2% for outdoor to achieve the state of the art. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_12745 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ProtoDepth: Unsupervised Continual Depth Completion with Prototypes Rim, Patrick Park, Hyoungseob Gangopadhyay, S. Zeng, Ziyao Chung, Younjoon Wong, Alex Computer Vision and Pattern Recognition We present ProtoDepth, a novel prototype-based approach for continual learning of unsupervised depth completion, the multimodal 3D reconstruction task of predicting dense depth maps from RGB images and sparse point clouds. The unsupervised learning paradigm is well-suited for continual learning, as ground truth is not needed. However, when training on new non-stationary distributions, depth completion models will catastrophically forget previously learned information. We address forgetting by learning prototype sets that adapt the latent features of a frozen pretrained model to new domains. Since the original weights are not modified, ProtoDepth does not forget when test-time domain identity is known. To extend ProtoDepth to the challenging setting where the test-time domain identity is withheld, we propose to learn domain descriptors that enable the model to select the appropriate prototype set for inference. We evaluate ProtoDepth on benchmark dataset sequences, where we reduce forgetting compared to baselines by 52.2% for indoor and 53.2% for outdoor to achieve the state of the art. |
| title | ProtoDepth: Unsupervised Continual Depth Completion with Prototypes |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.12745 |