Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918082284355584 |
|---|---|
| author | Feng, Zhenyang Wang, Zihe Gu, Jianyang Bueno, Saul Ibaven Frelek, Tomasz Ramesh, Advikaa Bai, Jingyan Wang, Lemeng Huang, Zanming Yoo, Jinsu Pan, Tai-Yu Chowdhury, Arpita Ramirez, Michelle Campolongo, Elizabeth G. Thompson, Matthew J. Lawrence, Christopher G. Record, Sydne Rosser, Neil Karpatne, Anuj Rubenstein, Daniel Lapp, Hilmar Stewart, Charles V. Berger-Wolf, Tanya Su, Yu Chao, Wei-Lun |
| author_facet | Feng, Zhenyang Wang, Zihe Gu, Jianyang Bueno, Saul Ibaven Frelek, Tomasz Ramesh, Advikaa Bai, Jingyan Wang, Lemeng Huang, Zanming Yoo, Jinsu Pan, Tai-Yu Chowdhury, Arpita Ramirez, Michelle Campolongo, Elizabeth G. Thompson, Matthew J. Lawrence, Christopher G. Record, Sydne Rosser, Neil Karpatne, Anuj Rubenstein, Daniel Lapp, Hilmar Stewart, Charles V. Berger-Wolf, Tanya Su, Yu Chao, Wei-Lun |
| contents | We study image segmentation in the biological domain, particularly trait segmentation from specimen images (e.g., butterfly wing stripes, beetle elytra). This fine-grained task is crucial for understanding the biology of organisms, but it traditionally requires manually annotating segmentation masks for hundreds of images per species, making it highly labor-intensive. To address this challenge, we propose a label-efficient approach, Static Segmentation by Tracking (SST), based on a key insight: while specimens of the same species exhibit natural variation, the traits of interest show up consistently. This motivates us to concatenate specimen images into a ``pseudo-video'' and reframe trait segmentation as a tracking problem. Specifically, SST generates masks for unlabeled images by propagating annotated or predicted masks from the ``pseudo-preceding'' images. Built upon recent video segmentation models, such as Segment Anything Model 2, SST achieves high-quality trait segmentation with only one labeled image per species, marking a breakthrough in specimen image analysis. To further enhance segmentation quality, we introduce a cycle-consistent loss for fine-tuning, again requiring only one labeled image. Additionally, we demonstrate the broader potential of SST, including one-shot instance segmentation in natural images and trait-based image retrieval. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_06749 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation Feng, Zhenyang Wang, Zihe Gu, Jianyang Bueno, Saul Ibaven Frelek, Tomasz Ramesh, Advikaa Bai, Jingyan Wang, Lemeng Huang, Zanming Yoo, Jinsu Pan, Tai-Yu Chowdhury, Arpita Ramirez, Michelle Campolongo, Elizabeth G. Thompson, Matthew J. Lawrence, Christopher G. Record, Sydne Rosser, Neil Karpatne, Anuj Rubenstein, Daniel Lapp, Hilmar Stewart, Charles V. Berger-Wolf, Tanya Su, Yu Chao, Wei-Lun Computer Vision and Pattern Recognition Artificial Intelligence We study image segmentation in the biological domain, particularly trait segmentation from specimen images (e.g., butterfly wing stripes, beetle elytra). This fine-grained task is crucial for understanding the biology of organisms, but it traditionally requires manually annotating segmentation masks for hundreds of images per species, making it highly labor-intensive. To address this challenge, we propose a label-efficient approach, Static Segmentation by Tracking (SST), based on a key insight: while specimens of the same species exhibit natural variation, the traits of interest show up consistently. This motivates us to concatenate specimen images into a ``pseudo-video'' and reframe trait segmentation as a tracking problem. Specifically, SST generates masks for unlabeled images by propagating annotated or predicted masks from the ``pseudo-preceding'' images. Built upon recent video segmentation models, such as Segment Anything Model 2, SST achieves high-quality trait segmentation with only one labeled image per species, marking a breakthrough in specimen image analysis. To further enhance segmentation quality, we introduce a cycle-consistent loss for fine-tuning, again requiring only one labeled image. Additionally, we demonstrate the broader potential of SST, including one-shot instance segmentation in natural images and trait-based image retrieval. |
| title | Static Segmentation by Tracking: A Label-Efficient Approach for Fine-Grained Specimen Image Segmentation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2501.06749 |