Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918042266501120 |
|---|---|
| author | Kierans, Aidan Ghosh, Avijit Hazan, Hananel Dori-Hacohen, Shiri |
| author_facet | Kierans, Aidan Ghosh, Avijit Hazan, Hananel Dori-Hacohen, Shiri |
| contents | Existing work on the alignment problem has focused mainly on (1) qualitative descriptions of the alignment problem; (2) attempting to align AI actions with human interests by focusing on value specification and learning; and/or (3) focusing on a single agent or on humanity as a monolith. Recent sociotechnical approaches highlight the need to understand complex misalignment among multiple human and AI agents. We address this gap by adapting a computational social science model of human contention to the alignment problem. Our model quantifies misalignment in large, diverse agent groups with potentially conflicting goals across various problem areas. Misalignment scores in our framework depend on the observed agent population, the domain in question, and conflict between agents' weighted preferences. Through simulations, we demonstrate how our model captures intuitive aspects of misalignment across different scenarios. We then apply our model to two case studies, including an autonomous vehicle setting, showcasing its practical utility. Our approach offers enhanced explanatory power for complex sociotechnical environments and could inform the design of more aligned AI systems in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_04231 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment Kierans, Aidan Ghosh, Avijit Hazan, Hananel Dori-Hacohen, Shiri Multiagent Systems Artificial Intelligence Computers and Society Computer Science and Game Theory I.2.11; K.4.m Existing work on the alignment problem has focused mainly on (1) qualitative descriptions of the alignment problem; (2) attempting to align AI actions with human interests by focusing on value specification and learning; and/or (3) focusing on a single agent or on humanity as a monolith. Recent sociotechnical approaches highlight the need to understand complex misalignment among multiple human and AI agents. We address this gap by adapting a computational social science model of human contention to the alignment problem. Our model quantifies misalignment in large, diverse agent groups with potentially conflicting goals across various problem areas. Misalignment scores in our framework depend on the observed agent population, the domain in question, and conflict between agents' weighted preferences. Through simulations, we demonstrate how our model captures intuitive aspects of misalignment across different scenarios. We then apply our model to two case studies, including an autonomous vehicle setting, showcasing its practical utility. Our approach offers enhanced explanatory power for complex sociotechnical environments and could inform the design of more aligned AI systems in real-world applications. |
| title | Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment |
| topic | Multiagent Systems Artificial Intelligence Computers and Society Computer Science and Game Theory I.2.11; K.4.m |
| url | https://arxiv.org/abs/2406.04231 |