PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914851171860480 |
|---|---|
| author | Zeng, Kuo-Hao Zhang, Zichen Ehsani, Kiana Hendrix, Rose Salvador, Jordi Herrasti, Alvaro Girshick, Ross Kembhavi, Aniruddha Weihs, Luca |
| author_facet | Zeng, Kuo-Hao Zhang, Zichen Ehsani, Kiana Hendrix, Rose Salvador, Jordi Herrasti, Alvaro Girshick, Ross Kembhavi, Aniruddha Weihs, Luca |
| contents | We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_20083 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators Zeng, Kuo-Hao Zhang, Zichen Ehsani, Kiana Hendrix, Rose Salvador, Jordi Herrasti, Alvaro Girshick, Ross Kembhavi, Aniruddha Weihs, Luca Robotics Computer Vision and Pattern Recognition We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning. |
| title | PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.20083 |