Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.02858 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913571889217536 |
|---|---|
| author | Gupta, Pranav Singh, Rishubh Shenoy, Pradeep Sarvadevabhatla, Ravikiran |
| author_facet | Gupta, Pranav Singh, Rishubh Shenoy, Pradeep Sarvadevabhatla, Ravikiran |
| contents | Multi-object multi-part scene segmentation is a challenging task whose complexity scales exponentially with part granularity and number of scene objects. To address the task, we propose a plug-and-play approach termed OLAF. First, we augment the input (RGB) with channels containing object-based structural cues (fg/bg mask, boundary edge mask). We propose a weight adaptation technique which enables regular (RGB) pre-trained models to process the augmented (5-channel) input in a stable manner during optimization. In addition, we introduce an encoder module termed LDF to provide low-level dense feature guidance. This assists segmentation, particularly for smaller parts. OLAF enables significant mIoU gains of $\mathbf{3.3}$ (Pascal-Parts-58), $\mathbf{3.5}$ (Pascal-Parts-108) over the SOTA model. On the most challenging variant (Pascal-Parts-201), the gain is $\mathbf{4.0}$. Experimentally, we show that OLAF's broad applicability enables gains across multiple architectures (CNN, U-Net, Transformer) and datasets. The code is available at olafseg.github.io |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_02858 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing Gupta, Pranav Singh, Rishubh Shenoy, Pradeep Sarvadevabhatla, Ravikiran Computer Vision and Pattern Recognition Multi-object multi-part scene segmentation is a challenging task whose complexity scales exponentially with part granularity and number of scene objects. To address the task, we propose a plug-and-play approach termed OLAF. First, we augment the input (RGB) with channels containing object-based structural cues (fg/bg mask, boundary edge mask). We propose a weight adaptation technique which enables regular (RGB) pre-trained models to process the augmented (5-channel) input in a stable manner during optimization. In addition, we introduce an encoder module termed LDF to provide low-level dense feature guidance. This assists segmentation, particularly for smaller parts. OLAF enables significant mIoU gains of $\mathbf{3.3}$ (Pascal-Parts-58), $\mathbf{3.5}$ (Pascal-Parts-108) over the SOTA model. On the most challenging variant (Pascal-Parts-201), the gain is $\mathbf{4.0}$. Experimentally, we show that OLAF's broad applicability enables gains across multiple architectures (CNN, U-Net, Transformer) and datasets. The code is available at olafseg.github.io |
| title | OLAF: A Plug-and-Play Framework for Enhanced Multi-object Multi-part Scene Parsing |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2411.02858 |