Visual Language Models as Operator Agents in the Space Domain
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913647920414720 |
|---|---|
| author | Carrasco, Alejandro Nedungadi, Marco Zucchelli, Enrico M. Jain, Amit Rodriguez-Fernandez, Victor Linares, Richard |
| author_facet | Carrasco, Alejandro Nedungadi, Marco Zucchelli, Enrico M. Jain, Amit Rodriguez-Fernandez, Victor Linares, Richard |
| contents | This paper explores the application of Vision-Language Models (VLMs) as operator agents in the space domain, focusing on both software and hardware operational paradigms. Building on advances in Large Language Models (LLMs) and their multimodal extensions, we investigate how VLMs can enhance autonomous control and decision-making in space missions. In the software context, we employ VLMs within the Kerbal Space Program Differential Games (KSPDG) simulation environment, enabling the agent to interpret visual screenshots of the graphical user interface to perform complex orbital maneuvers. In the hardware context, we integrate VLMs with robotic systems equipped with cameras to inspect and diagnose physical space objects, such as satellites. Our results demonstrate that VLMs can effectively process visual and textual data to generate contextually appropriate actions, competing with traditional methods and non-multimodal LLMs in simulation tasks, and showing promise in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_07802 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Visual Language Models as Operator Agents in the Space Domain Carrasco, Alejandro Nedungadi, Marco Zucchelli, Enrico M. Jain, Amit Rodriguez-Fernandez, Victor Linares, Richard Artificial Intelligence Space Physics This paper explores the application of Vision-Language Models (VLMs) as operator agents in the space domain, focusing on both software and hardware operational paradigms. Building on advances in Large Language Models (LLMs) and their multimodal extensions, we investigate how VLMs can enhance autonomous control and decision-making in space missions. In the software context, we employ VLMs within the Kerbal Space Program Differential Games (KSPDG) simulation environment, enabling the agent to interpret visual screenshots of the graphical user interface to perform complex orbital maneuvers. In the hardware context, we integrate VLMs with robotic systems equipped with cameras to inspect and diagnose physical space objects, such as satellites. Our results demonstrate that VLMs can effectively process visual and textual data to generate contextually appropriate actions, competing with traditional methods and non-multimodal LLMs in simulation tasks, and showing promise in real-world applications. |
| title | Visual Language Models as Operator Agents in the Space Domain |
| topic | Artificial Intelligence Space Physics |
| url | https://arxiv.org/abs/2501.07802 |