Minion: A Technology Probe to Explore How Users Negotiate Harmful Value Conflicts with AI Companions
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917231545286656 |
|---|---|
| author | Fan, Xianzhe Xiao, Qing Zhou, Xuhui Su, Yuran Lu, Zhicong Sap, Maarten Shen, Hong |
| author_facet | Fan, Xianzhe Xiao, Qing Zhou, Xuhui Su, Yuran Lu, Zhicong Sap, Maarten Shen, Hong |
| contents | AI companions are designed to foster emotionally engaging interactions, yet users often encounter conflicts that feel frustrating or hurtful, such as discriminatory statements and controlling behavior. This paper examines how users negotiate such harmful conflicts with AI companions and what emotional and practical burdens are created when mitigation is pushed to user-side tools. We analyze 146 public posts describing harmful value conflicts interacting with AI companions. We then introduce Minion, a Chrome-based technology probe that offers candidate responses spanning persuasion, rational appeals, boundary setting, and appeals to platform rules. Findings from a one-week probe study with 22 experienced users show how participants combine strategies, how emotional attachment motivates repair, and where conflicts become non-negotiable due to companion personas or platform policies. We surface design tensions in supporting value negotiation, showing how companion design can make some conflicts impossible to repair in practice, and derive implications for AI companion and support-tool design that caution against offloading safety work onto users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_07042 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Minion: A Technology Probe to Explore How Users Negotiate Harmful Value Conflicts with AI Companions Fan, Xianzhe Xiao, Qing Zhou, Xuhui Su, Yuran Lu, Zhicong Sap, Maarten Shen, Hong Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society AI companions are designed to foster emotionally engaging interactions, yet users often encounter conflicts that feel frustrating or hurtful, such as discriminatory statements and controlling behavior. This paper examines how users negotiate such harmful conflicts with AI companions and what emotional and practical burdens are created when mitigation is pushed to user-side tools. We analyze 146 public posts describing harmful value conflicts interacting with AI companions. We then introduce Minion, a Chrome-based technology probe that offers candidate responses spanning persuasion, rational appeals, boundary setting, and appeals to platform rules. Findings from a one-week probe study with 22 experienced users show how participants combine strategies, how emotional attachment motivates repair, and where conflicts become non-negotiable due to companion personas or platform policies. We surface design tensions in supporting value negotiation, showing how companion design can make some conflicts impossible to repair in practice, and derive implications for AI companion and support-tool design that caution against offloading safety work onto users. |
| title | Minion: A Technology Probe to Explore How Users Negotiate Harmful Value Conflicts with AI Companions |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2411.07042 |