At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909230605271040 |
|---|---|
| author | Tanoglidis, Dimitrios Jain, Bhuvnesh |
| author_facet | Tanoglidis, Dimitrios Jain, Bhuvnesh |
| contents | Vision-Language multimodal Models (VLMs) offer the possibility for zero-shot classification in astronomy: i.e. classification via natural language prompts, with no training. We investigate two models, GPT-4o and LLaVA-NeXT, for zero-shot classification of low-surface brightness galaxies and artifacts, as well as morphological classification of galaxies. We show that with natural language prompts these models achieved significant accuracy (above 80 percent typically) without additional training/fine tuning. We discuss areas that require improvement, especially for LLaVA-NeXT, which is an open source model. Our findings aim to motivate the astronomical community to consider VLMs as a powerful tool for both research and pedagogy, with the prospect that future custom-built or fine-tuned models could perform better. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_17057 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models Tanoglidis, Dimitrios Jain, Bhuvnesh Instrumentation and Methods for Astrophysics Astrophysics of Galaxies Artificial Intelligence Vision-Language multimodal Models (VLMs) offer the possibility for zero-shot classification in astronomy: i.e. classification via natural language prompts, with no training. We investigate two models, GPT-4o and LLaVA-NeXT, for zero-shot classification of low-surface brightness galaxies and artifacts, as well as morphological classification of galaxies. We show that with natural language prompts these models achieved significant accuracy (above 80 percent typically) without additional training/fine tuning. We discuss areas that require improvement, especially for LLaVA-NeXT, which is an open source model. Our findings aim to motivate the astronomical community to consider VLMs as a powerful tool for both research and pedagogy, with the prospect that future custom-built or fine-tuned models could perform better. |
| title | At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models |
| topic | Instrumentation and Methods for Astrophysics Astrophysics of Galaxies Artificial Intelligence |
| url | https://arxiv.org/abs/2406.17057 |