Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.15356 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911066930282496 |
|---|---|
| author | Guo, Lu Shan, Yixiang Zhu, Zhengbang Liang, Qifan Song, Lichang Long, Ting Zhang, Weinan Chang, Yi |
| author_facet | Guo, Lu Shan, Yixiang Zhu, Zhengbang Liang, Qifan Song, Lichang Long, Ting Zhang, Weinan Chang, Yi |
| contents | Offline reinforcement learning (RL) enables agents to learn policies from fixed datasets, avoiding costly or unsafe environment interactions. However, its effectiveness is often limited by dataset sparsity and the lack of transition overlap between suboptimal and expert trajectories, which makes long-horizon planning particularly challenging. Prior solutions based on synthetic data augmentation or trajectory stitching often fail to generalize to novel states and rely on heuristic stitching points. To address these challenges, we propose Retrieval High-quAlity Demonstrations (RAD) for decision-making, which combines non-parametric retrieval with diffusion-based generative modeling. RAD dynamically retrieves high-return states from the offline dataset as target states based on state similarity and return estimation, and plans toward them using a condition-guided diffusion model. Such retrieval-guided generation enables flexible trajectory stitching and improves generalization when encountered with underrepresented or out-of-distribution states. Extensive experiments confirm that RAD achieves competitive or superior performance compared to baselines across diverse benchmarks, validating its effectiveness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15356 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RAD: Retrieval High-quality Demonstrations to Enhance Decision-making Guo, Lu Shan, Yixiang Zhu, Zhengbang Liang, Qifan Song, Lichang Long, Ting Zhang, Weinan Chang, Yi Artificial Intelligence Offline reinforcement learning (RL) enables agents to learn policies from fixed datasets, avoiding costly or unsafe environment interactions. However, its effectiveness is often limited by dataset sparsity and the lack of transition overlap between suboptimal and expert trajectories, which makes long-horizon planning particularly challenging. Prior solutions based on synthetic data augmentation or trajectory stitching often fail to generalize to novel states and rely on heuristic stitching points. To address these challenges, we propose Retrieval High-quAlity Demonstrations (RAD) for decision-making, which combines non-parametric retrieval with diffusion-based generative modeling. RAD dynamically retrieves high-return states from the offline dataset as target states based on state similarity and return estimation, and plans toward them using a condition-guided diffusion model. Such retrieval-guided generation enables flexible trajectory stitching and improves generalization when encountered with underrepresented or out-of-distribution states. Extensive experiments confirm that RAD achieves competitive or superior performance compared to baselines across diverse benchmarks, validating its effectiveness. |
| title | RAD: Retrieval High-quality Demonstrations to Enhance Decision-making |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2507.15356 |