Human Preferences for Constructive Interactions in Language Model Alignment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915206696796160 |
|---|---|
| author | Kyrychenko, Yara Roozenbeek, Jon Davidson, Brandon van der Linden, Sander Debnath, Ramit |
| author_facet | Kyrychenko, Yara Roozenbeek, Jon Davidson, Brandon van der Linden, Sander Debnath, Ramit |
| contents | As large language models (LLMs) enter the mainstream, aligning them to foster constructive dialogue rather than exacerbate societal divisions is critical. Using an individualized and multicultural alignment dataset of over 7,500 conversations of individuals from 74 countries engaging with 21 LLMs, we examined how linguistic attributes linked to constructive interactions are reflected in human preference data used for training AI. We found that users consistently preferred well-reasoned and nuanced responses while rejecting those high in personal storytelling. However, users who believed that AI should reflect their values tended to place less preference on reasoning in LLM responses and more on curiosity. Encouragingly, we observed that users could set the tone for how constructive their conversation would be, as LLMs mirrored linguistic attributes, including toxicity, in user queries. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16480 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Human Preferences for Constructive Interactions in Language Model Alignment Kyrychenko, Yara Roozenbeek, Jon Davidson, Brandon van der Linden, Sander Debnath, Ramit Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society As large language models (LLMs) enter the mainstream, aligning them to foster constructive dialogue rather than exacerbate societal divisions is critical. Using an individualized and multicultural alignment dataset of over 7,500 conversations of individuals from 74 countries engaging with 21 LLMs, we examined how linguistic attributes linked to constructive interactions are reflected in human preference data used for training AI. We found that users consistently preferred well-reasoned and nuanced responses while rejecting those high in personal storytelling. However, users who believed that AI should reflect their values tended to place less preference on reasoning in LLM responses and more on curiosity. Encouragingly, we observed that users could set the tone for how constructive their conversation would be, as LLMs mirrored linguistic attributes, including toxicity, in user queries. |
| title | Human Preferences for Constructive Interactions in Language Model Alignment |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2503.16480 |