CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911793297752064 |
|---|---|
| author | Qi, Siyuan Chen, Shuo Li, Yexin Kong, Xiangyu Wang, Junqi Yang, Bangcheng Wong, Pring Zhong, Yifan Zhang, Xiaoyuan Zhang, Zhaowei Liu, Nian Wang, Wei Yang, Yaodong Zhu, Song-Chun |
| author_facet | Qi, Siyuan Chen, Shuo Li, Yexin Kong, Xiangyu Wang, Junqi Yang, Bangcheng Wong, Pring Zhong, Yifan Zhang, Xiaoyuan Zhang, Zhaowei Liu, Nian Wang, Wei Yang, Yaodong Zhu, Song-Chun |
| contents | The generalization of decision-making agents encompasses two fundamental elements: learning from past experiences and reasoning in novel contexts. However, the predominant emphasis in most interactive environments is on learning, often at the expense of complexity in reasoning. In this paper, we introduce CivRealm, an environment inspired by the Civilization game. Civilization's profound alignment with human history and society necessitates sophisticated learning, while its ever-changing situations demand strong reasoning to generalize. Particularly, CivRealm sets up an imperfect-information general-sum game with a changing number of players; it presents a plethora of complex features, challenging the agent to deal with open-ended stochastic environments that require diplomacy and negotiation skills. Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning. To catalyze further research, we present initial results for both paradigms. The canonical RL-based agents exhibit reasonable performance in mini-games, whereas both RL- and LLM-based agents struggle to make substantial progress in the full game. Overall, CivRealm stands as a unique learning and reasoning challenge for decision-making agents. The code is available at https://github.com/bigai-ai/civrealm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10568 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents Qi, Siyuan Chen, Shuo Li, Yexin Kong, Xiangyu Wang, Junqi Yang, Bangcheng Wong, Pring Zhong, Yifan Zhang, Xiaoyuan Zhang, Zhaowei Liu, Nian Wang, Wei Yang, Yaodong Zhu, Song-Chun Artificial Intelligence The generalization of decision-making agents encompasses two fundamental elements: learning from past experiences and reasoning in novel contexts. However, the predominant emphasis in most interactive environments is on learning, often at the expense of complexity in reasoning. In this paper, we introduce CivRealm, an environment inspired by the Civilization game. Civilization's profound alignment with human history and society necessitates sophisticated learning, while its ever-changing situations demand strong reasoning to generalize. Particularly, CivRealm sets up an imperfect-information general-sum game with a changing number of players; it presents a plethora of complex features, challenging the agent to deal with open-ended stochastic environments that require diplomacy and negotiation skills. Within CivRealm, we provide interfaces for two typical agent types: tensor-based agents that focus on learning, and language-based agents that emphasize reasoning. To catalyze further research, we present initial results for both paradigms. The canonical RL-based agents exhibit reasonable performance in mini-games, whereas both RL- and LLM-based agents struggle to make substantial progress in the full game. Overall, CivRealm stands as a unique learning and reasoning challenge for decision-making agents. The code is available at https://github.com/bigai-ai/civrealm. |
| title | CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2401.10568 |