Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916901212389376 |
|---|---|
| author | Yang, Xuning Eppner, Clemens Tremblay, Jonathan Fox, Dieter Birchfield, Stan Ramos, Fabio |
| author_facet | Yang, Xuning Eppner, Clemens Tremblay, Jonathan Fox, Dieter Birchfield, Stan Ramos, Fabio |
| contents | Current vision-based robotics simulation benchmarks have significantly advanced robotic manipulation research. However, robotics is fundamentally a real-world problem, and evaluation for real-world applications has lagged behind in evaluating generalist policies. In this paper, we discuss challenges and desiderata in designing benchmarks for generalist robotic manipulation policies for the goal of sim-to-real policy transfer. We propose 1) utilizing high visual-fidelity simulation for improved sim-to-real transfer, 2) evaluating policies by systematically increasing task complexity and scenario perturbation to assess robustness, and 3) quantifying performance alignment between real-world performance and its simulation counterparts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11117 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective Yang, Xuning Eppner, Clemens Tremblay, Jonathan Fox, Dieter Birchfield, Stan Ramos, Fabio Robotics Current vision-based robotics simulation benchmarks have significantly advanced robotic manipulation research. However, robotics is fundamentally a real-world problem, and evaluation for real-world applications has lagged behind in evaluating generalist policies. In this paper, we discuss challenges and desiderata in designing benchmarks for generalist robotic manipulation policies for the goal of sim-to-real policy transfer. We propose 1) utilizing high visual-fidelity simulation for improved sim-to-real transfer, 2) evaluating policies by systematically increasing task complexity and scenario perturbation to assess robustness, and 3) quantifying performance alignment between real-world performance and its simulation counterparts. |
| title | Robot Policy Evaluation for Sim-to-Real Transfer: A Benchmarking Perspective |
| topic | Robotics |
| url | https://arxiv.org/abs/2508.11117 |