Towards LLM Agents for Earth Observation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918140251734016 |
|---|---|
| author | Kao, Chia Hsiang Zhao, Wenting Revankar, Shreelekha Speas, Samuel Bhagat, Snehal Datta, Rajeev Phoo, Cheng Perng Mall, Utkarsh Vondrick, Carl Bala, Kavita Hariharan, Bharath |
| author_facet | Kao, Chia Hsiang Zhao, Wenting Revankar, Shreelekha Speas, Samuel Bhagat, Snehal Datta, Rajeev Phoo, Cheng Perng Mall, Utkarsh Vondrick, Carl Bala, Kavita Hariharan, Bharath |
| contents | Earth Observation (EO) provides critical planetary data for environmental monitoring, disaster management, climate science, and other scientific domains. Here we ask: Are AI systems ready for reliable Earth Observation? We introduce \datasetnamenospace, a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors. Using Google Earth Engine API as a tool, LLM agents can only achieve an accuracy of 33% because the code fails to run over 58% of the time. We improve the failure rate for open models by fine-tuning synthetic data, allowing much smaller models (Llama-3.1-8B) to achieve comparable accuracy to much larger ones (e.g., DeepSeek-R1). Taken together, our findings identify significant challenges to be solved before AI agents can automate earth observation, and suggest paths forward. The project page is available at https://iandrover.github.io/UnivEarth. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_12110 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards LLM Agents for Earth Observation Kao, Chia Hsiang Zhao, Wenting Revankar, Shreelekha Speas, Samuel Bhagat, Snehal Datta, Rajeev Phoo, Cheng Perng Mall, Utkarsh Vondrick, Carl Bala, Kavita Hariharan, Bharath Artificial Intelligence Earth Observation (EO) provides critical planetary data for environmental monitoring, disaster management, climate science, and other scientific domains. Here we ask: Are AI systems ready for reliable Earth Observation? We introduce \datasetnamenospace, a benchmark of 140 yes/no questions from NASA Earth Observatory articles across 13 topics and 17 satellite sensors. Using Google Earth Engine API as a tool, LLM agents can only achieve an accuracy of 33% because the code fails to run over 58% of the time. We improve the failure rate for open models by fine-tuning synthetic data, allowing much smaller models (Llama-3.1-8B) to achieve comparable accuracy to much larger ones (e.g., DeepSeek-R1). Taken together, our findings identify significant challenges to be solved before AI agents can automate earth observation, and suggest paths forward. The project page is available at https://iandrover.github.io/UnivEarth. |
| title | Towards LLM Agents for Earth Observation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2504.12110 |