Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.21057 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916811498323968 |
|---|---|
| author | Miao, Zhuochen Lv, Jun Fang, Hongjie Jin, Yang Lu, Cewu |
| author_facet | Miao, Zhuochen Lv, Jun Fang, Hongjie Jin, Yang Lu, Cewu |
| contents | Imitation learning has emerged as a powerful paradigm in robot manipulation, yet its generalization capability remains constrained by object-specific dependencies in limited expert demonstrations. To address this challenge, we propose knowledge-driven imitation learning, a framework that leverages external structural semantic knowledge to abstract object representations within the same category. We introduce a novel semantic keypoint graph as a knowledge template and develop a coarse-to-fine template-matching algorithm that optimizes both structural consistency and semantic similarity. Evaluated on three real-world robotic manipulation tasks, our method achieves superior performance, surpassing image-based diffusion policies with only one-quarter of the expert demonstrations. Extensive experiments further demonstrate its robustness across novel objects, backgrounds, and lighting conditions. This work pioneers a knowledge-driven approach to data-efficient robotic learning in real-world settings. Code and more materials are available on https://knowledge-driven.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_21057 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions Miao, Zhuochen Lv, Jun Fang, Hongjie Jin, Yang Lu, Cewu Robotics Imitation learning has emerged as a powerful paradigm in robot manipulation, yet its generalization capability remains constrained by object-specific dependencies in limited expert demonstrations. To address this challenge, we propose knowledge-driven imitation learning, a framework that leverages external structural semantic knowledge to abstract object representations within the same category. We introduce a novel semantic keypoint graph as a knowledge template and develop a coarse-to-fine template-matching algorithm that optimizes both structural consistency and semantic similarity. Evaluated on three real-world robotic manipulation tasks, our method achieves superior performance, surpassing image-based diffusion policies with only one-quarter of the expert demonstrations. Extensive experiments further demonstrate its robustness across novel objects, backgrounds, and lighting conditions. This work pioneers a knowledge-driven approach to data-efficient robotic learning in real-world settings. Code and more materials are available on https://knowledge-driven.github.io/. |
| title | Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions |
| topic | Robotics |
| url | https://arxiv.org/abs/2506.21057 |