Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.16939 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912081784078336 |
|---|---|
| author | Heinemann, Lena Jaus, Alexander Marinov, Zdravko Kim, Moon Spadea, Maria Francesca Kleesiek, Jens Stiefelhagen, Rainer |
| author_facet | Heinemann, Lena Jaus, Alexander Marinov, Zdravko Kim, Moon Spadea, Maria Francesca Kleesiek, Jens Stiefelhagen, Rainer |
| contents | Within this work, we introduce LIMIS: The first purely language-based interactive medical image segmentation model. We achieve this by adapting Grounded SAM to the medical domain and designing a language-based model interaction strategy that allows radiologists to incorporate their knowledge into the segmentation process. LIMIS produces high-quality initial segmentation masks by leveraging medical foundation models and allows users to adapt segmentation masks using only language, opening up interactive segmentation to scenarios where physicians require using their hands for other tasks. We evaluate LIMIS on three publicly available medical datasets in terms of performance and usability with experts from the medical domain confirming its high-quality segmentation masks and its interactive usability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16939 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LIMIS: Towards Language-based Interactive Medical Image Segmentation Heinemann, Lena Jaus, Alexander Marinov, Zdravko Kim, Moon Spadea, Maria Francesca Kleesiek, Jens Stiefelhagen, Rainer Computer Vision and Pattern Recognition Within this work, we introduce LIMIS: The first purely language-based interactive medical image segmentation model. We achieve this by adapting Grounded SAM to the medical domain and designing a language-based model interaction strategy that allows radiologists to incorporate their knowledge into the segmentation process. LIMIS produces high-quality initial segmentation masks by leveraging medical foundation models and allows users to adapt segmentation masks using only language, opening up interactive segmentation to scenarios where physicians require using their hands for other tasks. We evaluate LIMIS on three publicly available medical datasets in terms of performance and usability with experts from the medical domain confirming its high-quality segmentation masks and its interactive usability. |
| title | LIMIS: Towards Language-based Interactive Medical Image Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.16939 |