World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917956110254080 |
|---|---|
| author | Wang, Siyin Fei, Zhaoye Cheng, Qinyuan Zhang, Shiduo Cai, Panpan Fu, Jinlan Qiu, Xipeng |
| author_facet | Wang, Siyin Fei, Zhaoye Cheng, Qinyuan Zhang, Shiduo Cai, Panpan Fu, Jinlan Qiu, Xipeng |
| contents | Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or leverage world models during inference, overlooking the benefits of learning to model the world as a way to enhance planning capabilities. We propose Dual Preference Optimization (D$^2$PO), a new learning framework that jointly optimizes state prediction and action selection through preference learning, enabling LVLMs to understand environment dynamics for better planning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Extensive experiments on VoTa-Bench demonstrate that our D$^2$PO-based method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B), achieving superior task success rates with more efficient execution paths. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10480 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning Wang, Siyin Fei, Zhaoye Cheng, Qinyuan Zhang, Shiduo Cai, Panpan Fu, Jinlan Qiu, Xipeng Computation and Language Computer Vision and Pattern Recognition Robotics Recent advances in large vision-language models (LVLMs) have shown promise for embodied task planning, yet they struggle with fundamental challenges like dependency constraints and efficiency. Existing approaches either solely optimize action selection or leverage world models during inference, overlooking the benefits of learning to model the world as a way to enhance planning capabilities. We propose Dual Preference Optimization (D$^2$PO), a new learning framework that jointly optimizes state prediction and action selection through preference learning, enabling LVLMs to understand environment dynamics for better planning. To automatically collect trajectories and stepwise preference data without human annotation, we introduce a tree search mechanism for extensive exploration via trial-and-error. Extensive experiments on VoTa-Bench demonstrate that our D$^2$PO-based method significantly outperforms existing methods and GPT-4o when applied to Qwen2-VL (7B), LLaVA-1.6 (7B), and LLaMA-3.2 (11B), achieving superior task success rates with more efficient execution paths. |
| title | World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning |
| topic | Computation and Language Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2503.10480 |