Boosting Vision-Language Models for Histopathology Classification: Predict all at once
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910588455616512 |
|---|---|
| author | Zanella, Maxime Shakeri, Fereshteh Huang, Yunshi Bahig, Houda Ayed, Ismail Ben |
| author_facet | Zanella, Maxime Shakeri, Fereshteh Huang, Yunshi Bahig, Houda Ayed, Ismail Ben |
| contents | The development of vision-language models (VLMs) for histo-pathology has shown promising new usages and zero-shot performances. However, current approaches, which decompose large slides into smaller patches, focus solely on inductive classification, i.e., prediction for each patch is made independently of the other patches in the target test data. We extend the capability of these large models by introducing a transductive approach. By using text-based predictions and affinity relationships among patches, our approach leverages the strong zero-shot capabilities of these new VLMs without any additional labels. Our experiments cover four histopathology datasets and five different VLMs. Operating solely in the embedding space (i.e., in a black-box setting), our approach is highly efficient, processing $10^5$ patches in just a few seconds, and shows significant accuracy improvements over inductive zero-shot classification. Code available at https://github.com/FereshteShakeri/Histo-TransCLIP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_01883 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Boosting Vision-Language Models for Histopathology Classification: Predict all at once Zanella, Maxime Shakeri, Fereshteh Huang, Yunshi Bahig, Houda Ayed, Ismail Ben Computer Vision and Pattern Recognition The development of vision-language models (VLMs) for histo-pathology has shown promising new usages and zero-shot performances. However, current approaches, which decompose large slides into smaller patches, focus solely on inductive classification, i.e., prediction for each patch is made independently of the other patches in the target test data. We extend the capability of these large models by introducing a transductive approach. By using text-based predictions and affinity relationships among patches, our approach leverages the strong zero-shot capabilities of these new VLMs without any additional labels. Our experiments cover four histopathology datasets and five different VLMs. Operating solely in the embedding space (i.e., in a black-box setting), our approach is highly efficient, processing $10^5$ patches in just a few seconds, and shows significant accuracy improvements over inductive zero-shot classification. Code available at https://github.com/FereshteShakeri/Histo-TransCLIP. |
| title | Boosting Vision-Language Models for Histopathology Classification: Predict all at once |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.01883 |