Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.05360 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917887835373568 |
|---|---|
| author | Dable-Heath, Edmund Vodenicharski, Boyko Bishop, James |
| author_facet | Dable-Heath, Edmund Vodenicharski, Boyko Bishop, James |
| contents | Corrigibility of autonomous agents is an under explored part of system design, with previous work focusing on single agent systems. It has been suggested that uncertainty over the human preferences acts to keep the agents corrigible, even in the face of human irrationality. We present a general framework for modelling corrigibility in a multi-agent setting as a 2 player game in which the agents always have a move in which they can ask the human for supervision. This is formulated as a Bayesian game for the purpose of introducing uncertainty over the human beliefs. We further analyse two specific cases. First, a two player corrigibility game, in which we want corrigibility displayed in both agents for both common payoff (monotone) games and harmonic games. Then we investigate an adversary setting, in which one agent is considered to be a `defending' agent and the other an `adversary'. A general result is provided for what belief over the games and human rationality the defending agent is required to have to induce corrigibility. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05360 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Corrigibility and Alignment in Multi Agent Games Dable-Heath, Edmund Vodenicharski, Boyko Bishop, James Computer Science and Game Theory Artificial Intelligence Corrigibility of autonomous agents is an under explored part of system design, with previous work focusing on single agent systems. It has been suggested that uncertainty over the human preferences acts to keep the agents corrigible, even in the face of human irrationality. We present a general framework for modelling corrigibility in a multi-agent setting as a 2 player game in which the agents always have a move in which they can ask the human for supervision. This is formulated as a Bayesian game for the purpose of introducing uncertainty over the human beliefs. We further analyse two specific cases. First, a two player corrigibility game, in which we want corrigibility displayed in both agents for both common payoff (monotone) games and harmonic games. Then we investigate an adversary setting, in which one agent is considered to be a `defending' agent and the other an `adversary'. A general result is provided for what belief over the games and human rationality the defending agent is required to have to induce corrigibility. |
| title | On Corrigibility and Alignment in Multi Agent Games |
| topic | Computer Science and Game Theory Artificial Intelligence |
| url | https://arxiv.org/abs/2501.05360 |