Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911422563221504 |
|---|---|
| author | Xu, Hengxuan Lan, Fengbo Zhao, Zhixin Wang, Shengjie Liu, Mengqiao Sun, Jieqian Cheng, Yu Zhang, Tao |
| author_facet | Xu, Hengxuan Lan, Fengbo Zhao, Zhixin Wang, Shengjie Liu, Mengqiao Sun, Jieqian Cheng, Yu Zhang, Tao |
| contents | Enabling robots to explore and act in unfamiliar environments under ambiguous human instructions by interactively identifying task-relevant objects (e.g., identifying cups or beverages for "I'm thirsty") remains challenging for existing vision-language model (VLM)-based methods. This challenge stems from inefficient reasoning and the lack of environmental interaction, which hinder real-time task planning and execution. To address this, We propose Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions (AIDE), a dual-stream framework that integrates interactive exploration with vision-language reasoning, where Multi-Stage Inference (MSI) serves as the decision-making stream and Accelerated Decision-Making (ADM) as the execution stream, enabling zero-shot affordance analysis and interpretation of ambiguous instructions. Extensive experiments in simulation and real-world environments show that AIDE achieves the task planning success rate of over 80\% and more than 95\% accuracy in closed-loop continuous execution at 10 Hz, outperforming existing VLM-based methods in diverse open-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_05273 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions Xu, Hengxuan Lan, Fengbo Zhao, Zhixin Wang, Shengjie Liu, Mengqiao Sun, Jieqian Cheng, Yu Zhang, Tao Robotics I.2.9; I.2.6 Enabling robots to explore and act in unfamiliar environments under ambiguous human instructions by interactively identifying task-relevant objects (e.g., identifying cups or beverages for "I'm thirsty") remains challenging for existing vision-language model (VLM)-based methods. This challenge stems from inefficient reasoning and the lack of environmental interaction, which hinder real-time task planning and execution. To address this, We propose Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions (AIDE), a dual-stream framework that integrates interactive exploration with vision-language reasoning, where Multi-Stage Inference (MSI) serves as the decision-making stream and Accelerated Decision-Making (ADM) as the execution stream, enabling zero-shot affordance analysis and interpretation of ambiguous instructions. Extensive experiments in simulation and real-world environments show that AIDE achieves the task planning success rate of over 80\% and more than 95\% accuracy in closed-loop continuous execution at 10 Hz, outperforming existing VLM-based methods in diverse open-world scenarios. |
| title | Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions |
| topic | Robotics I.2.9; I.2.6 |
| url | https://arxiv.org/abs/2602.05273 |