Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929570689581056 |
|---|---|
| author | Shi, Chengshuai Yang, Kun Yang, Jing Shen, Cong |
| author_facet | Shi, Chengshuai Yang, Kun Yang, Jing Shen, Cong |
| contents | The in-context learning (ICL) capability of pre-trained models based on the transformer architecture has received growing interest in recent years. While theoretical understanding has been obtained for ICL in reinforcement learning (RL), the previous results are largely confined to the single-agent setting. This work proposes to further explore the in-context learning capabilities of pre-trained transformer models in competitive multi-agent games, i.e., in-context game-playing (ICGP). Focusing on the classical two-player zero-sum games, theoretical guarantees are provided to demonstrate that pre-trained transformers can provably learn to approximate Nash equilibrium in an in-context manner for both decentralized and centralized learning settings. As a key part of the proof, constructional results are established to demonstrate that the transformer architecture is sufficiently rich to realize celebrated multi-agent game-playing algorithms, in particular, decentralized V-learning and centralized VI-ULCB. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09701 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models Shi, Chengshuai Yang, Kun Yang, Jing Shen, Cong Machine Learning Computer Science and Game Theory Information Theory Multiagent Systems The in-context learning (ICL) capability of pre-trained models based on the transformer architecture has received growing interest in recent years. While theoretical understanding has been obtained for ICL in reinforcement learning (RL), the previous results are largely confined to the single-agent setting. This work proposes to further explore the in-context learning capabilities of pre-trained transformer models in competitive multi-agent games, i.e., in-context game-playing (ICGP). Focusing on the classical two-player zero-sum games, theoretical guarantees are provided to demonstrate that pre-trained transformers can provably learn to approximate Nash equilibrium in an in-context manner for both decentralized and centralized learning settings. As a key part of the proof, constructional results are established to demonstrate that the transformer architecture is sufficiently rich to realize celebrated multi-agent game-playing algorithms, in particular, decentralized V-learning and centralized VI-ULCB. |
| title | Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models |
| topic | Machine Learning Computer Science and Game Theory Information Theory Multiagent Systems |
| url | https://arxiv.org/abs/2410.09701 |