iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912511375179776 |
|---|---|
| author | Wang, Kohou Liu, ZhaoXiang Bai, Lin Fan, Kun Liu, Xiang Hu, Huan Wang, Kai Lian, Shiguo |
| author_facet | Wang, Kohou Liu, ZhaoXiang Bai, Lin Fan, Kun Liu, Xiang Hu, Huan Wang, Kai Lian, Shiguo |
| contents | It is crucial that robots' performance can be improved after deployment, as they are inherently likely to encounter novel scenarios never seen before. This paper presents an innovative solution: an interactive learning-based robot system powered by a Multi-modal Large Language Model(MLLM). A key feature of our system is its ability to learn from natural dialogues with non-expert users. We also propose chain of question to clarify the exact intent of the question before providing an answer and dual-modality retrieval modules to leverage these interaction events to avoid repeating same mistakes, ensuring a seamless user experience before model updates, which is in contrast to current mainstream MLLM-based robotic systems. Our system marks a novel approach in robotics by integrating interactive learning, paving the way for superior adaptability and performance in diverse environments. We demonstrate the effectiveness and improvement of our method through experiments, both quantitively and qualitatively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_22896 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement Wang, Kohou Liu, ZhaoXiang Bai, Lin Fan, Kun Liu, Xiang Hu, Huan Wang, Kai Lian, Shiguo Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Robotics It is crucial that robots' performance can be improved after deployment, as they are inherently likely to encounter novel scenarios never seen before. This paper presents an innovative solution: an interactive learning-based robot system powered by a Multi-modal Large Language Model(MLLM). A key feature of our system is its ability to learn from natural dialogues with non-expert users. We also propose chain of question to clarify the exact intent of the question before providing an answer and dual-modality retrieval modules to leverage these interaction events to avoid repeating same mistakes, ensuring a seamless user experience before model updates, which is in contrast to current mainstream MLLM-based robotic systems. Our system marks a novel approach in robotics by integrating interactive learning, paving the way for superior adaptability and performance in diverse environments. We demonstrate the effectiveness and improvement of our method through experiments, both quantitively and qualitatively. |
| title | iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement |
| topic | Human-Computer Interaction Artificial Intelligence Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2507.22896 |