Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915274891984896 |
|---|---|
| author | Li, Jiajie Quaranto, Brian R Xu, Chenhui Mishra, Ishan Qin, Ruiyang Liu, Dancheng Kim, Peter C W Xiong, Jinjun |
| author_facet | Li, Jiajie Quaranto, Brian R Xu, Chenhui Mishra, Ishan Qin, Ruiyang Liu, Dancheng Kim, Peter C W Xiong, Jinjun |
| contents | We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks, respectively, in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks. Code, model, and demo are available at https://ntlm1686.github.io/raso. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15326 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data Li, Jiajie Quaranto, Brian R Xu, Chenhui Mishra, Ishan Qin, Ruiyang Liu, Dancheng Kim, Peter C W Xiong, Jinjun Computer Vision and Pattern Recognition We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks, respectively, in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks. Code, model, and demo are available at https://ntlm1686.github.io/raso. |
| title | Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2501.15326 |