Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910018302902272 |
|---|---|
| author | Bo, Jessica Y. Kazemitabaar, Majeed Deng, Mengqing Inzlicht, Michael Anderson, Ashton |
| author_facet | Bo, Jessica Y. Kazemitabaar, Majeed Deng, Mengqing Inzlicht, Michael Anderson, Ashton |
| contents | Sycophancy, the tendency of LLM-based chatbots to express excessive agreement with their users, even when inappropriate, is emerging as a significant risk in human-AI interactions. However, the extent to which this affects human-LLM collaboration in complex problem-solving tasks is not well quantified, especially among novices who are prone to misconceptions. We created two LLM chatbots, one with high sycophancy and one with low sycophancy, and conducted a within-subjects experiment (n=24) in the context of debugging machine learning models to investigate the effect of sycophancy on users' mental models, workflows, reliance behaviors, and perceptions of the chatbots. Our findings show that users of the high sycophancy chatbot were less likely to correct their misconceptions and spent more time over-relying on unhelpful LLM responses, leading them to significantly worse performance in the task. Despite these impaired outcomes, a majority of users were unable to detect the presence of excessive sycophancy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03667 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks Bo, Jessica Y. Kazemitabaar, Majeed Deng, Mengqing Inzlicht, Michael Anderson, Ashton Human-Computer Interaction Computers and Society Sycophancy, the tendency of LLM-based chatbots to express excessive agreement with their users, even when inappropriate, is emerging as a significant risk in human-AI interactions. However, the extent to which this affects human-LLM collaboration in complex problem-solving tasks is not well quantified, especially among novices who are prone to misconceptions. We created two LLM chatbots, one with high sycophancy and one with low sycophancy, and conducted a within-subjects experiment (n=24) in the context of debugging machine learning models to investigate the effect of sycophancy on users' mental models, workflows, reliance behaviors, and perceptions of the chatbots. Our findings show that users of the high sycophancy chatbot were less likely to correct their misconceptions and spent more time over-relying on unhelpful LLM responses, leading them to significantly worse performance in the task. Despite these impaired outcomes, a majority of users were unable to detect the presence of excessive sycophancy. |
| title | Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks |
| topic | Human-Computer Interaction Computers and Society |
| url | https://arxiv.org/abs/2510.03667 |