ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflection
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911007736070144 |
|---|---|
| author | Tsai, Shang-Chi Kawano, Seiya Contreras, Angel Garcia Yoshino, Koichiro Chen, Yun-Nung |
| author_facet | Tsai, Shang-Chi Kawano, Seiya Contreras, Angel Garcia Yoshino, Koichiro Chen, Yun-Nung |
| contents | When designing robots to assist in everyday human activities, it is crucial to enhance user requests with visual cues from their surroundings for improved intent understanding. This process is defined as a multimodal classification task. However, gathering a large-scale dataset encompassing both visual and linguistic elements for model training is challenging and time-consuming. To address this issue, our paper introduces a novel framework focusing on data augmentation in robotic assistance scenarios, encompassing both dialogues and related environmental imagery. This approach involves leveraging a sophisticated large language model to simulate potential conversations and environmental contexts, followed by the use of a stable diffusion model to create images depicting these environments. The additionally generated data serves to refine the latest multimodal models, enabling them to more accurately determine appropriate actions in response to user interactions with the limited target data. Our experimental results, based on a dataset collected from real-world scenarios, demonstrate that our methodology significantly enhances the robot's action selection capabilities, achieving the state-of-the-art performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13956 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflection Tsai, Shang-Chi Kawano, Seiya Contreras, Angel Garcia Yoshino, Koichiro Chen, Yun-Nung Computation and Language Artificial Intelligence Robotics When designing robots to assist in everyday human activities, it is crucial to enhance user requests with visual cues from their surroundings for improved intent understanding. This process is defined as a multimodal classification task. However, gathering a large-scale dataset encompassing both visual and linguistic elements for model training is challenging and time-consuming. To address this issue, our paper introduces a novel framework focusing on data augmentation in robotic assistance scenarios, encompassing both dialogues and related environmental imagery. This approach involves leveraging a sophisticated large language model to simulate potential conversations and environmental contexts, followed by the use of a stable diffusion model to create images depicting these environments. The additionally generated data serves to refine the latest multimodal models, enabling them to more accurately determine appropriate actions in response to user interactions with the limited target data. Our experimental results, based on a dataset collected from real-world scenarios, demonstrate that our methodology significantly enhances the robot's action selection capabilities, achieving the state-of-the-art performance. |
| title | ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflection |
| topic | Computation and Language Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2506.13956 |