Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2510.01051 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910036303806464 |
|---|---|
| author | Liu, Zichen Sims, Anya Duan, Keyu Chen, Changyu Yu, Simon Zhou, Xiangxin Xu, Haotian Xiong, Shaopan Liu, Bo Tan, Chenmien Beh, Chuen Yang Wang, Weixun Zhu, Hao Shi, Weiyan Yang, Diyi Shieh, Michael Teh, Yee Whye Lee, Wee Sun Lin, Min |
| author_facet | Liu, Zichen Sims, Anya Duan, Keyu Chen, Changyu Yu, Simon Zhou, Xiangxin Xu, Haotian Xiong, Shaopan Liu, Bo Tan, Chenmien Beh, Chuen Yang Wang, Weixun Zhu, Hao Shi, Weiyan Yang, Diyi Shieh, Michael Teh, Yee Whye Lee, Wee Sun Lin, Min |
| contents | The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environments. To facilitate this transition we introduce GEM (General Experience Maker), an open-source environment simulator designed for the age of LLMs. Analogous to OpenAI-Gym for traditional reinforcement learning (RL), GEM provides a standardized framework for the environment-agent interface, including asynchronous vectorized execution for high throughput, and flexible wrappers for easy extensibility. GEM also features a diverse suite of environments, robust integrated tools, and single-file example scripts demonstrating using GEM with five popular RL training frameworks. Along with this, we also provide a set of baselines across 24 environments using REINFORCE with Return Batch Normalization (ReBN), which -- unlike GRPO -- is compatible with the full RL setting of dense per-turn rewards and offers better credit assignment. We further conduct apple-to-apple benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings using GEM to shed light on the algorithmic designs. Lastly, GEM also functions as a convenient evaluation toolkit besides a training environment. We hope this framework can help accelerate future agentic LLM research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01051 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GEM: A Gym for Agentic LLMs Liu, Zichen Sims, Anya Duan, Keyu Chen, Changyu Yu, Simon Zhou, Xiangxin Xu, Haotian Xiong, Shaopan Liu, Bo Tan, Chenmien Beh, Chuen Yang Wang, Weixun Zhu, Hao Shi, Weiyan Yang, Diyi Shieh, Michael Teh, Yee Whye Lee, Wee Sun Lin, Min Machine Learning Artificial Intelligence Computation and Language The training paradigm for large language models (LLMs) is moving from static datasets to experience-based learning, where agents acquire skills via interacting with complex environments. To facilitate this transition we introduce GEM (General Experience Maker), an open-source environment simulator designed for the age of LLMs. Analogous to OpenAI-Gym for traditional reinforcement learning (RL), GEM provides a standardized framework for the environment-agent interface, including asynchronous vectorized execution for high throughput, and flexible wrappers for easy extensibility. GEM also features a diverse suite of environments, robust integrated tools, and single-file example scripts demonstrating using GEM with five popular RL training frameworks. Along with this, we also provide a set of baselines across 24 environments using REINFORCE with Return Batch Normalization (ReBN), which -- unlike GRPO -- is compatible with the full RL setting of dense per-turn rewards and offers better credit assignment. We further conduct apple-to-apple benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings using GEM to shed light on the algorithmic designs. Lastly, GEM also functions as a convenient evaluation toolkit besides a training environment. We hope this framework can help accelerate future agentic LLM research. |
| title | GEM: A Gym for Agentic LLMs |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2510.01051 |