Flow Policy Gradients for Robot Control
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912869078007808 |
|---|---|
| author | Yi, Brent Choi, Hongsuk Singh, Himanshu Gaurav Huang, Xiaoyu Truong, Takara E. Sferrazza, Carmelo Ma, Yi Duan, Rocky Abbeel, Pieter Shi, Guanya Liu, Karen Kanazawa, Angjoo |
| author_facet | Yi, Brent Choi, Hongsuk Singh, Himanshu Gaurav Huang, Xiaoyu Truong, Takara E. Sferrazza, Carmelo Ma, Yi Duan, Rocky Abbeel, Pieter Shi, Guanya Liu, Karen Kanazawa, Angjoo |
| contents | Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how flow matching policy gradients -- a recent framework that bypasses likelihood computation -- can be made effective for training and fine-tuning more expressive policies in challenging robot control settings. We introduce an improved objective that enables success in legged locomotion, humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved fine-tuning robustness over baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02481 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Flow Policy Gradients for Robot Control Yi, Brent Choi, Hongsuk Singh, Himanshu Gaurav Huang, Xiaoyu Truong, Takara E. Sferrazza, Carmelo Ma, Yi Duan, Rocky Abbeel, Pieter Shi, Guanya Liu, Karen Kanazawa, Angjoo Robotics Artificial Intelligence Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how flow matching policy gradients -- a recent framework that bypasses likelihood computation -- can be made effective for training and fine-tuning more expressive policies in challenging robot control settings. We introduce an improved objective that enables success in legged locomotion, humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved fine-tuning robustness over baselines. |
| title | Flow Policy Gradients for Robot Control |
| topic | Robotics Artificial Intelligence |
| url | https://arxiv.org/abs/2602.02481 |