CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917690318258176 |
|---|---|
| author | Pei, Jialun Jiang, Tao Tang, He Liu, Nian Jin, Yueming Fan, Deng-Ping Heng, Pheng-Ann |
| author_facet | Pei, Jialun Jiang, Tao Tang, He Liu, Nian Jin, Yueming Fan, Deng-Ping Heng, Pheng-Ann |
| contents | We propose a novel approach for RGB-D salient instance segmentation using a dual-branch cross-modal feature calibration architecture called CalibNet. Our method simultaneously calibrates depth and RGB features in the kernel and mask branches to generate instance-aware kernels and mask features. CalibNet consists of three simple modules, a dynamic interactive kernel (DIK) and a weight-sharing fusion (WSF), which work together to generate effective instance-aware kernels and integrate cross-modal features. To improve the quality of depth features, we incorporate a depth similarity assessment (DSA) module prior to DIK and WSF. In addition, we further contribute a new DSIS dataset, which contains 1,940 images with elaborate instance-level annotations. Extensive experiments on three challenging benchmarks show that CalibNet yields a promising result, i.e., 58.0% AP with 320*480 input size on the COME15K-N test set, which significantly surpasses the alternative frameworks. Our code and dataset are available at: https://github.com/PJLallen/CalibNet. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_08098 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation Pei, Jialun Jiang, Tao Tang, He Liu, Nian Jin, Yueming Fan, Deng-Ping Heng, Pheng-Ann Computer Vision and Pattern Recognition We propose a novel approach for RGB-D salient instance segmentation using a dual-branch cross-modal feature calibration architecture called CalibNet. Our method simultaneously calibrates depth and RGB features in the kernel and mask branches to generate instance-aware kernels and mask features. CalibNet consists of three simple modules, a dynamic interactive kernel (DIK) and a weight-sharing fusion (WSF), which work together to generate effective instance-aware kernels and integrate cross-modal features. To improve the quality of depth features, we incorporate a depth similarity assessment (DSA) module prior to DIK and WSF. In addition, we further contribute a new DSIS dataset, which contains 1,940 images with elaborate instance-level annotations. Extensive experiments on three challenging benchmarks show that CalibNet yields a promising result, i.e., 58.0% AP with 320*480 input size on the COME15K-N test set, which significantly surpasses the alternative frameworks. Our code and dataset are available at: https://github.com/PJLallen/CalibNet. |
| title | CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2307.08098 |