Steering Language Models with Game-Theoretic Solvers
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909429051424768 |
|---|---|
| author | Gemp, Ian Patel, Roma Bachrach, Yoram Lanctot, Marc Dasagi, Vibhavari Marris, Luke Piliouras, Georgios Liu, Siqi Tuyls, Karl |
| author_facet | Gemp, Ian Patel, Roma Bachrach, Yoram Lanctot, Marc Dasagi, Vibhavari Marris, Luke Piliouras, Georgios Liu, Siqi Tuyls, Karl |
| contents | Mathematical models of interactions among rational agents have long been studied in game theory. However these interactions are often over a small set of discrete game actions which is very different from how humans communicate in natural language. To bridge this gap, we introduce a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs). Specifically, by modelling the players, strategies and payoffs in a "game" of dialogue, we create a binding from natural language interactions to the conventional symbolic logic of game theory. Given this binding, we can ask existing game-theoretic algorithms to provide us with strategic solutions (e.g., what string an LLM should generate to maximize payoff in the face of strategic partners or opponents), giving us predictors of stable, rational conversational strategies. We focus on three domains that require different negotiation strategies: scheduling meetings, trading fruit and debate, and evaluate an LLM's generated language when guided by solvers. We see that LLMs that follow game-theory solvers result in dialogue generations that are less exploitable than the control (no guidance from solvers), and the language generated results in higher rewards, in all negotiation domains. We discuss future implications of this work, and how game-theoretic solvers that can leverage the expressivity of natural language can open up a new avenue of guiding language research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01704 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Steering Language Models with Game-Theoretic Solvers Gemp, Ian Patel, Roma Bachrach, Yoram Lanctot, Marc Dasagi, Vibhavari Marris, Luke Piliouras, Georgios Liu, Siqi Tuyls, Karl Computation and Language Artificial Intelligence Computer Science and Game Theory Mathematical models of interactions among rational agents have long been studied in game theory. However these interactions are often over a small set of discrete game actions which is very different from how humans communicate in natural language. To bridge this gap, we introduce a framework that allows equilibrium solvers to work over the space of natural language dialogue generated by large language models (LLMs). Specifically, by modelling the players, strategies and payoffs in a "game" of dialogue, we create a binding from natural language interactions to the conventional symbolic logic of game theory. Given this binding, we can ask existing game-theoretic algorithms to provide us with strategic solutions (e.g., what string an LLM should generate to maximize payoff in the face of strategic partners or opponents), giving us predictors of stable, rational conversational strategies. We focus on three domains that require different negotiation strategies: scheduling meetings, trading fruit and debate, and evaluate an LLM's generated language when guided by solvers. We see that LLMs that follow game-theory solvers result in dialogue generations that are less exploitable than the control (no guidance from solvers), and the language generated results in higher rewards, in all negotiation domains. We discuss future implications of this work, and how game-theoretic solvers that can leverage the expressivity of natural language can open up a new avenue of guiding language research. |
| title | Steering Language Models with Game-Theoretic Solvers |
| topic | Computation and Language Artificial Intelligence Computer Science and Game Theory |
| url | https://arxiv.org/abs/2402.01704 |