Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916592580820992 |
|---|---|
| author | Zang, Ying Cao, Runlong Zhang, Jianqi Han, Yidong Cao, Ziyue Hu, Wenjun Zhu, Didi Zhu, Lanyun Li, Zejian Ji, Deyi Chen, Tianrun |
| author_facet | Zang, Ying Cao, Runlong Zhang, Jianqi Han, Yidong Cao, Ziyue Hu, Wenjun Zhu, Didi Zhu, Lanyun Li, Zejian Ji, Deyi Chen, Tianrun |
| contents | Sketches, with their expressive potential, allow humans to convey the essence of an object through even a rough contour. For the first time, we harness this expressive potential to improve segmentation performance in challenging tasks like camouflaged object detection (COD). Our approach introduces an innovative sketch-guided interactive segmentation framework, allowing users to intuitively annotate objects with freehand sketches (drawing a rough contour of the object) instead of the traditional bounding boxes or points used in classic interactive segmentation models like SAM. We demonstrate that sketch input can significantly improve performance in existing iterative segmentation methods, outperforming text or bounding box annotations. Additionally, we introduce key modifications to network architectures and a novel sketch augmentation technique to fully harness the power of sketch input and further boost segmentation accuracy. Remarkably, our model' s output can be directly used to train other neural networks, achieving results comparable to pixel-by-pixel annotations--while reducing annotation time by up to 120 times, which shows great potential in democratizing the annotation process and enabling model training with less reliance on resource-intensive, laborious pixel-level annotations. We also present KOSCamo+, the first freehand sketch dataset for camouflaged object detection. The dataset, code, and the labeling tool will be open sourced. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_19329 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches Zang, Ying Cao, Runlong Zhang, Jianqi Han, Yidong Cao, Ziyue Hu, Wenjun Zhu, Didi Zhu, Lanyun Li, Zejian Ji, Deyi Chen, Tianrun Computer Vision and Pattern Recognition Sketches, with their expressive potential, allow humans to convey the essence of an object through even a rough contour. For the first time, we harness this expressive potential to improve segmentation performance in challenging tasks like camouflaged object detection (COD). Our approach introduces an innovative sketch-guided interactive segmentation framework, allowing users to intuitively annotate objects with freehand sketches (drawing a rough contour of the object) instead of the traditional bounding boxes or points used in classic interactive segmentation models like SAM. We demonstrate that sketch input can significantly improve performance in existing iterative segmentation methods, outperforming text or bounding box annotations. Additionally, we introduce key modifications to network architectures and a novel sketch augmentation technique to fully harness the power of sketch input and further boost segmentation accuracy. Remarkably, our model' s output can be directly used to train other neural networks, achieving results comparable to pixel-by-pixel annotations--while reducing annotation time by up to 120 times, which shows great potential in democratizing the annotation process and enabling model training with less reliance on resource-intensive, laborious pixel-level annotations. We also present KOSCamo+, the first freehand sketch dataset for camouflaged object detection. The dataset, code, and the labeling tool will be open sourced. |
| title | Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.19329 |