Don't lie to your friends: Learning what you know from collaborative self-play
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915469463650304 |
|---|---|
| author | Eisenstein, Jacob Aghajani, Reza Fisch, Adam Dua, Dheeru Huot, Fantine Lapata, Mirella Zayats, Vicky Berant, Jonathan |
| author_facet | Eisenstein, Jacob Aghajani, Reza Fisch, Adam Dua, Dheeru Huot, Fantine Lapata, Mirella Zayats, Vicky Berant, Jonathan |
| contents | To be helpful assistants, AI agents must be aware of their own capabilities and limitations. This includes knowing when to answer from parametric knowledge versus using tools, when to trust tool outputs, and when to abstain or hedge. Such capabilities are hard to teach through supervised fine-tuning because they require constructing examples that reflect the agent's specific capabilities. We therefore propose a radically new approach to teaching agents what they know: \emph{collaborative self-play}. We construct multi-agent collaborations in which the group is rewarded for collectively arriving at correct answers. The desired meta-knowledge emerges from the incentives built into the structure of the interaction. We focus on small societies of agents that have access to heterogeneous tools (corpus-specific retrieval), and therefore must collaborate to maximize their success while minimizing their effort. Experiments show that group-level rewards for multi-agent communities can induce policies that \emph{transfer} to improve tool use and selective prediction in settings where individual agents are deployed in isolation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14481 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Don't lie to your friends: Learning what you know from collaborative self-play Eisenstein, Jacob Aghajani, Reza Fisch, Adam Dua, Dheeru Huot, Fantine Lapata, Mirella Zayats, Vicky Berant, Jonathan Machine Learning Computation and Language To be helpful assistants, AI agents must be aware of their own capabilities and limitations. This includes knowing when to answer from parametric knowledge versus using tools, when to trust tool outputs, and when to abstain or hedge. Such capabilities are hard to teach through supervised fine-tuning because they require constructing examples that reflect the agent's specific capabilities. We therefore propose a radically new approach to teaching agents what they know: \emph{collaborative self-play}. We construct multi-agent collaborations in which the group is rewarded for collectively arriving at correct answers. The desired meta-knowledge emerges from the incentives built into the structure of the interaction. We focus on small societies of agents that have access to heterogeneous tools (corpus-specific retrieval), and therefore must collaborate to maximize their success while minimizing their effort. Experiments show that group-level rewards for multi-agent communities can induce policies that \emph{transfer} to improve tool use and selective prediction in settings where individual agents are deployed in isolation. |
| title | Don't lie to your friends: Learning what you know from collaborative self-play |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2503.14481 |