Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.03399 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911299697377280 |
|---|---|
| author | Edelman, Joe Zhi-Xuan, Tan Lowe, Ryan Klingefjord, Oliver Wang-Mascianica, Vincent Franklin, Matija Kearns, Ryan Othniel Hain, Ellie Sarkar, Atrisha Bakker, Michiel Barez, Fazl Duvenaud, David Foerster, Jakob Gabriel, Iason Gubbels, Joseph Goodman, Bryce Haupt, Andreas Heitzig, Jobst Jara-Ettinger, Julian Kasirzadeh, Atoosa Kirkpatrick, James Ravi Koh, Andrew Knox, W. Bradley Koralus, Philipp Lehman, Joel Levine, Sydney Marro, Samuele Revel, Manon Shorin, Toby Sutherland, Morgan Tessler, Michael Henry Vendrov, Ivan Wilken-Smith, James |
| author_facet | Edelman, Joe Zhi-Xuan, Tan Lowe, Ryan Klingefjord, Oliver Wang-Mascianica, Vincent Franklin, Matija Kearns, Ryan Othniel Hain, Ellie Sarkar, Atrisha Bakker, Michiel Barez, Fazl Duvenaud, David Foerster, Jakob Gabriel, Iason Gubbels, Joseph Goodman, Bryce Haupt, Andreas Heitzig, Jobst Jara-Ettinger, Julian Kasirzadeh, Atoosa Kirkpatrick, James Ravi Koh, Andrew Knox, W. Bradley Koralus, Philipp Lehman, Joel Levine, Sydney Marro, Samuele Revel, Manon Shorin, Toby Sutherland, Morgan Tessler, Michael Henry Vendrov, Ivan Wilken-Smith, James |
| contents | Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to the intentions of its operating organization can lead to bad outcomes if the goals of that organization are misaligned with those of other institutions and individuals. For this reason, we need full-stack alignment, the concurrent alignment of AI systems and the institutions that shape them with what people value. This can be done without imposing a particular vision of individual or collective flourishing. We argue that current approaches for representing values, such as utility functions, preference orderings, or unstructured text, struggle to address these and other issues effectively. They struggle to distinguish values from other signals, to support principled normative reasoning, and to model collective goods. We propose thick models of value will be needed. These structure the way values and norms are represented, enabling systems to distinguish enduring values from fleeting preferences, to model the social embedding of individual choices, and to reason normatively, applying values in new domains. We demonstrate this approach in five areas: AI value stewardship, normatively competent agents, win-win negotiation systems, meaning-preserving economic mechanisms, and democratic regulatory institutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_03399 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value Edelman, Joe Zhi-Xuan, Tan Lowe, Ryan Klingefjord, Oliver Wang-Mascianica, Vincent Franklin, Matija Kearns, Ryan Othniel Hain, Ellie Sarkar, Atrisha Bakker, Michiel Barez, Fazl Duvenaud, David Foerster, Jakob Gabriel, Iason Gubbels, Joseph Goodman, Bryce Haupt, Andreas Heitzig, Jobst Jara-Ettinger, Julian Kasirzadeh, Atoosa Kirkpatrick, James Ravi Koh, Andrew Knox, W. Bradley Koralus, Philipp Lehman, Joel Levine, Sydney Marro, Samuele Revel, Manon Shorin, Toby Sutherland, Morgan Tessler, Michael Henry Vendrov, Ivan Wilken-Smith, James Machine Learning Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to the intentions of its operating organization can lead to bad outcomes if the goals of that organization are misaligned with those of other institutions and individuals. For this reason, we need full-stack alignment, the concurrent alignment of AI systems and the institutions that shape them with what people value. This can be done without imposing a particular vision of individual or collective flourishing. We argue that current approaches for representing values, such as utility functions, preference orderings, or unstructured text, struggle to address these and other issues effectively. They struggle to distinguish values from other signals, to support principled normative reasoning, and to model collective goods. We propose thick models of value will be needed. These structure the way values and norms are represented, enabling systems to distinguish enduring values from fleeting preferences, to model the social embedding of individual choices, and to reason normatively, applying values in new domains. We demonstrate this approach in five areas: AI value stewardship, normatively competent agents, win-win negotiation systems, meaning-preserving economic mechanisms, and democratic regulatory institutions. |
| title | Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.03399 |