Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.17652 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913575008731136 |
|---|---|
| author | Sun, Fan-Yun Harini, S. I. Yi, Angela Zhou, Yihan Zook, Alex Tremblay, Jonathan Cross, Logan Wu, Jiajun Haber, Nick |
| author_facet | Sun, Fan-Yun Harini, S. I. Yi, Angela Zhou, Yihan Zook, Alex Tremblay, Jonathan Cross, Logan Wu, Jiajun Haber, Nick |
| contents | Generating simulations to train intelligent agents in game-playing and robotics from natural language input, from user input or task documentation, remains an open-ended challenge. Existing approaches focus on parts of this challenge, such as generating reward functions or task hyperparameters. Unlike previous work, we introduce FACTORSIM that generates full simulations in code from language input that can be used to train agents. Exploiting the structural modularity specific to coded simulations, we propose to use a factored partially observable Markov decision process representation that allows us to reduce context dependence during each step of the generation. For evaluation, we introduce a generative simulation benchmark that assesses the generated simulation code's accuracy and effectiveness in facilitating zero-shot transfers in reinforcement learning settings. We show that FACTORSIM outperforms existing methods in generating simulations regarding prompt alignment (e.g., accuracy), zero-shot transfer abilities, and human evaluation. We also demonstrate its effectiveness in generating robotic tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_17652 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FactorSim: Generative Simulation via Factorized Representation Sun, Fan-Yun Harini, S. I. Yi, Angela Zhou, Yihan Zook, Alex Tremblay, Jonathan Cross, Logan Wu, Jiajun Haber, Nick Artificial Intelligence Robotics Generating simulations to train intelligent agents in game-playing and robotics from natural language input, from user input or task documentation, remains an open-ended challenge. Existing approaches focus on parts of this challenge, such as generating reward functions or task hyperparameters. Unlike previous work, we introduce FACTORSIM that generates full simulations in code from language input that can be used to train agents. Exploiting the structural modularity specific to coded simulations, we propose to use a factored partially observable Markov decision process representation that allows us to reduce context dependence during each step of the generation. For evaluation, we introduce a generative simulation benchmark that assesses the generated simulation code's accuracy and effectiveness in facilitating zero-shot transfers in reinforcement learning settings. We show that FACTORSIM outperforms existing methods in generating simulations regarding prompt alignment (e.g., accuracy), zero-shot transfer abilities, and human evaluation. We also demonstrate its effectiveness in generating robotic tasks. |
| title | FactorSim: Generative Simulation via Factorized Representation |
| topic | Artificial Intelligence Robotics |
| url | https://arxiv.org/abs/2409.17652 |