Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915916943458304 |
|---|---|
| author | Zhang, Zhiping Zhang, Yi Evie Shi, Freda Li, Tianshi |
| author_facet | Zhang, Zhiping Zhang, Yi Evie Shi, Freda Li, Tianshi |
| contents | LLM agents require personal information for personalization in order to effectively act on users' behalf, but this raises privacy concerns that can discourage data sharing, limiting both the autonomy levels at which agents can operate and the effectiveness of personalization. Yet the expanded design space of agent autonomy also presents opportunities to shape these effects, which remain underexplored. We conducted a $3\times3$ between-subjects experiment ($N=450$) to study how agent autonomy level influences personalization's effects on users' privacy concerns, trust, and willingness to use, as well as the underlying psychological processes. We find that risk-contingent autonomy, where the agent delegates control to users upon detecting potential privacy leakage, through improving users' perceived control, attenuates personalization's adverse effects by reducing the increase in privacy concerns and the decrease in trust. Our results suggest that designing $\textbf{agent's autonomy}$ that supports $\textbf{human autonomy}$ (both in terms of perceived control and oversight effectiveness) helps users benefit from personalization without being deterred by growing privacy concerns, contributing to the development of trustworthy LLM agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_04465 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents Zhang, Zhiping Zhang, Yi Evie Shi, Freda Li, Tianshi Human-Computer Interaction Artificial Intelligence Cryptography and Security LLM agents require personal information for personalization in order to effectively act on users' behalf, but this raises privacy concerns that can discourage data sharing, limiting both the autonomy levels at which agents can operate and the effectiveness of personalization. Yet the expanded design space of agent autonomy also presents opportunities to shape these effects, which remain underexplored. We conducted a $3\times3$ between-subjects experiment ($N=450$) to study how agent autonomy level influences personalization's effects on users' privacy concerns, trust, and willingness to use, as well as the underlying psychological processes. We find that risk-contingent autonomy, where the agent delegates control to users upon detecting potential privacy leakage, through improving users' perceived control, attenuates personalization's adverse effects by reducing the increase in privacy concerns and the decrease in trust. Our results suggest that designing $\textbf{agent's autonomy}$ that supports $\textbf{human autonomy}$ (both in terms of perceived control and oversight effectiveness) helps users benefit from personalization without being deterred by growing privacy concerns, contributing to the development of trustworthy LLM agents. |
| title | Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents |
| topic | Human-Computer Interaction Artificial Intelligence Cryptography and Security |
| url | https://arxiv.org/abs/2510.04465 |