Societal Alignment Frameworks Can Improve LLM Alignment
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929735860224000 |
|---|---|
| author | Stańczak, Karolina Meade, Nicholas Bhatia, Mehar Zhou, Hattie Böttinger, Konstantin Barnes, Jeremy Stanley, Jason Montgomery, Jessica Zemel, Richard Papernot, Nicolas Chapados, Nicolas Therien, Denis Lillicrap, Timothy P. Marasović, Ana Delacroix, Sylvie Hadfield, Gillian K. Reddy, Siva |
| author_facet | Stańczak, Karolina Meade, Nicholas Bhatia, Mehar Zhou, Hattie Böttinger, Konstantin Barnes, Jeremy Stanley, Jason Montgomery, Jessica Zemel, Richard Papernot, Nicolas Chapados, Nicolas Therien, Denis Lillicrap, Timothy P. Marasović, Ana Delacroix, Sylvie Hadfield, Gillian K. Reddy, Siva |
| contents | Recent progress in large language models (LLMs) has focused on producing responses that meet human expectations and align with shared values - a process coined alignment. However, aligning LLMs remains challenging due to the inherent disconnect between the complexity of human values and the narrow nature of the technological approaches designed to address them. Current alignment methods often lead to misspecified objectives, reflecting the broader issue of incomplete contracts, the impracticality of specifying a contract between a model developer, and the model that accounts for every scenario in LLM alignment. In this paper, we argue that improving LLM alignment requires incorporating insights from societal alignment frameworks, including social, economic, and contractual alignment, and discuss potential solutions drawn from these domains. Given the role of uncertainty within societal alignment frameworks, we then investigate how it manifests in LLM alignment. We end our discussion by offering an alternative view on LLM alignment, framing the underspecified nature of its objectives as an opportunity rather than perfect their specification. Beyond technical improvements in LLM alignment, we discuss the need for participatory alignment interface designs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_00069 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Societal Alignment Frameworks Can Improve LLM Alignment Stańczak, Karolina Meade, Nicholas Bhatia, Mehar Zhou, Hattie Böttinger, Konstantin Barnes, Jeremy Stanley, Jason Montgomery, Jessica Zemel, Richard Papernot, Nicolas Chapados, Nicolas Therien, Denis Lillicrap, Timothy P. Marasović, Ana Delacroix, Sylvie Hadfield, Gillian K. Reddy, Siva Computers and Society Artificial Intelligence Computation and Language Recent progress in large language models (LLMs) has focused on producing responses that meet human expectations and align with shared values - a process coined alignment. However, aligning LLMs remains challenging due to the inherent disconnect between the complexity of human values and the narrow nature of the technological approaches designed to address them. Current alignment methods often lead to misspecified objectives, reflecting the broader issue of incomplete contracts, the impracticality of specifying a contract between a model developer, and the model that accounts for every scenario in LLM alignment. In this paper, we argue that improving LLM alignment requires incorporating insights from societal alignment frameworks, including social, economic, and contractual alignment, and discuss potential solutions drawn from these domains. Given the role of uncertainty within societal alignment frameworks, we then investigate how it manifests in LLM alignment. We end our discussion by offering an alternative view on LLM alignment, framing the underspecified nature of its objectives as an opportunity rather than perfect their specification. Beyond technical improvements in LLM alignment, we discuss the need for participatory alignment interface designs. |
| title | Societal Alignment Frameworks Can Improve LLM Alignment |
| topic | Computers and Society Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2503.00069 |