ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914134700851200 |
|---|---|
| author | Shen, Hua Knearem, Tiffany Ghosh, Reshmi Yang, Yu-Ju Clark, Nicholas Mitra, Tanushree Huang, Yun |
| author_facet | Shen, Hua Knearem, Tiffany Ghosh, Reshmi Yang, Yu-Ju Clark, Nicholas Mitra, Tanushree Huang, Yun |
| contents | As AI systems become more advanced, ensuring their alignment with a diverse range of individuals and societal values becomes increasingly critical. But how can we capture fundamental human values and assess the degree to which AI systems align with them? We introduce ValueCompass, a framework of fundamental values, grounded in psychological theory and a systematic review, to identify and evaluate human-AI alignment. We apply ValueCompass to measure the value alignment of humans and large language models (LLMs) across four real-world scenarios: collaborative writing, education, public sectors, and healthcare. Our findings reveal concerning misalignments between humans and LLMs, such as humans frequently endorse values like "National Security" which were largely rejected by LLMs. We also observe that values differ across scenarios, highlighting the need for context-aware AI alignment strategies. This work provides valuable insights into the design space of human-AI alignment, laying the foundations for developing AI systems that responsibly reflect societal values and ethics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_09586 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs Shen, Hua Knearem, Tiffany Ghosh, Reshmi Yang, Yu-Ju Clark, Nicholas Mitra, Tanushree Huang, Yun Human-Computer Interaction Artificial Intelligence Computation and Language As AI systems become more advanced, ensuring their alignment with a diverse range of individuals and societal values becomes increasingly critical. But how can we capture fundamental human values and assess the degree to which AI systems align with them? We introduce ValueCompass, a framework of fundamental values, grounded in psychological theory and a systematic review, to identify and evaluate human-AI alignment. We apply ValueCompass to measure the value alignment of humans and large language models (LLMs) across four real-world scenarios: collaborative writing, education, public sectors, and healthcare. Our findings reveal concerning misalignments between humans and LLMs, such as humans frequently endorse values like "National Security" which were largely rejected by LLMs. We also observe that values differ across scenarios, highlighting the need for context-aware AI alignment strategies. This work provides valuable insights into the design space of human-AI alignment, laying the foundations for developing AI systems that responsibly reflect societal values and ethics. |
| title | ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2409.09586 |