Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916011100340224 |
|---|---|
| author | Liang, Anthony Korkmaz, Yigit Zhang, Jiahui Hwang, Minyoung Anwar, Abrar Kaushik, Sidhant Shah, Aditya Huang, Alex S. Zettlemoyer, Luke Fox, Dieter Xiang, Yu Li, Anqi Bobu, Andreea Gupta, Abhishek Tu, Stephen Biyik, Erdem Zhang, Jesse |
| author_facet | Liang, Anthony Korkmaz, Yigit Zhang, Jiahui Hwang, Minyoung Anwar, Abrar Kaushik, Sidhant Shah, Aditya Huang, Alex S. Zettlemoyer, Luke Fox, Dieter Xiang, Yu Li, Anqi Bobu, Andreea Gupta, Abhishek Tu, Stephen Biyik, Erdem Zhang, Jesse |
| contents | General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations, providing only local, frame-level supervision. While effective for expert demonstrations, this paradigm scales poorly to large-scale robotics datasets where failed and suboptimal trajectories are abundant and assigning dense progress labels is ambiguous. We introduce Robometer, a scalable reward modeling framework that combines intra-trajectory progress supervision with inter-trajectory preference supervision. Robometer is trained with a dual objective: a frame-level progress loss that anchors reward magnitude on expert data, and a trajectory-comparison preference loss that imposes global ordering constraints across trajectories of the same task, enabling effective learning from both real and augmented failed trajectories. To support this formulation at scale, we curate RBM-1M, a reward-learning dataset comprising over one million trajectories spanning diverse robot embodiments and tasks, including substantial suboptimal and failure data. Across benchmarks and real-world evaluations, Robometer learns more generalizable reward functions than prior methods and improves robot learning performance across a diverse set of downstream applications. Code, model weights, and videos at https://robometer.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_02115 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons Liang, Anthony Korkmaz, Yigit Zhang, Jiahui Hwang, Minyoung Anwar, Abrar Kaushik, Sidhant Shah, Aditya Huang, Alex S. Zettlemoyer, Luke Fox, Dieter Xiang, Yu Li, Anqi Bobu, Andreea Gupta, Abhishek Tu, Stephen Biyik, Erdem Zhang, Jesse Robotics Artificial Intelligence Machine Learning General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations, providing only local, frame-level supervision. While effective for expert demonstrations, this paradigm scales poorly to large-scale robotics datasets where failed and suboptimal trajectories are abundant and assigning dense progress labels is ambiguous. We introduce Robometer, a scalable reward modeling framework that combines intra-trajectory progress supervision with inter-trajectory preference supervision. Robometer is trained with a dual objective: a frame-level progress loss that anchors reward magnitude on expert data, and a trajectory-comparison preference loss that imposes global ordering constraints across trajectories of the same task, enabling effective learning from both real and augmented failed trajectories. To support this formulation at scale, we curate RBM-1M, a reward-learning dataset comprising over one million trajectories spanning diverse robot embodiments and tasks, including substantial suboptimal and failure data. Across benchmarks and real-world evaluations, Robometer learns more generalizable reward functions than prior methods and improves robot learning performance across a diverse set of downstream applications. Code, model weights, and videos at https://robometer.github.io/. |
| title | Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2603.02115 |