CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929281877147648 |
|---|---|
| author | Lykov, Artem Konenkov, Mikhail Gbagbe, Koffivi Fidèle Litvinov, Mikhail Davletshin, Denis Fedoseev, Aleksey Cabrera, Miguel Altamirano Peter, Robinroy Tsetserukou, Dzmitry |
| author_facet | Lykov, Artem Konenkov, Mikhail Gbagbe, Koffivi Fidèle Litvinov, Mikhail Davletshin, Denis Fedoseev, Aleksey Cabrera, Miguel Altamirano Peter, Robinroy Tsetserukou, Dzmitry |
| contents | This paper introduces CognitiveOS, the first operating system designed for cognitive robots capable of functioning across diverse robotic platforms. CognitiveOS is structured as a multi-agent system comprising modules built upon a transformer architecture, facilitating communication through an internal monologue format. These modules collectively empower the robot to tackle intricate real-world tasks. The paper delineates the operational principles of the system along with descriptions of its nine distinct modules. The modular design endows the system with distinctive advantages over traditional end-to-end methodologies, notably in terms of adaptability and scalability. The system's modules are configurable, modifiable, or deactivatable depending on the task requirements, while new modules can be seamlessly integrated. This system serves as a foundational resource for researchers and developers in the cognitive robotics domain, alleviating the burden of constructing a cognitive robot system from scratch. Experimental findings demonstrate the system's advanced task comprehension and adaptability across varied tasks, robotic platforms, and module configurations, underscoring its potential for real-world applications. Moreover, in the category of Reasoning it outperformed CognitiveDog (by 15%) and RT2 (by 31%), achieving the highest to date rate of 77%. We provide a code repository and dataset for the replication of CognitiveOS: link will be provided in camera-ready submission. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_16205 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI Lykov, Artem Konenkov, Mikhail Gbagbe, Koffivi Fidèle Litvinov, Mikhail Davletshin, Denis Fedoseev, Aleksey Cabrera, Miguel Altamirano Peter, Robinroy Tsetserukou, Dzmitry Robotics This paper introduces CognitiveOS, the first operating system designed for cognitive robots capable of functioning across diverse robotic platforms. CognitiveOS is structured as a multi-agent system comprising modules built upon a transformer architecture, facilitating communication through an internal monologue format. These modules collectively empower the robot to tackle intricate real-world tasks. The paper delineates the operational principles of the system along with descriptions of its nine distinct modules. The modular design endows the system with distinctive advantages over traditional end-to-end methodologies, notably in terms of adaptability and scalability. The system's modules are configurable, modifiable, or deactivatable depending on the task requirements, while new modules can be seamlessly integrated. This system serves as a foundational resource for researchers and developers in the cognitive robotics domain, alleviating the burden of constructing a cognitive robot system from scratch. Experimental findings demonstrate the system's advanced task comprehension and adaptability across varied tasks, robotic platforms, and module configurations, underscoring its potential for real-world applications. Moreover, in the category of Reasoning it outperformed CognitiveDog (by 15%) and RT2 (by 31%), achieving the highest to date rate of 77%. We provide a code repository and dataset for the replication of CognitiveOS: link will be provided in camera-ready submission. |
| title | CognitiveOS: Large Multimodal Model based System to Endow Any Type of Robot with Generative AI |
| topic | Robotics |
| url | https://arxiv.org/abs/2401.16205 |