When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917301568143360 |
|---|---|
| author | Kumar, Harsh Chahal, Jasmine Zhao, Yinuo Zhang, Zeling Wei, Annika Tay, Louis Anderson, Ashton |
| author_facet | Kumar, Harsh Chahal, Jasmine Zhao, Yinuo Zhang, Zeling Wei, Annika Tay, Louis Anderson, Ashton |
| contents | Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agents and ecosystems blending AI, crowd input, and expert oversight. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08937 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being Kumar, Harsh Chahal, Jasmine Zhao, Yinuo Zhang, Zeling Wei, Annika Tay, Louis Anderson, Ashton Human-Computer Interaction Artificial Intelligence Computers and Society Seeking advice is a core human behavior that the internet has reinvented twice: first through forums and Q&A communities that crowdsource public guidance, and now through large language models (LLMs). Yet the quality of this LLM advice for everyday well-being scenarios remains unclear. How does it compare, not only against human comments, but against the wisdom of the online crowd? We ran two studies (N=210) in which experts compared top-voted Reddit advice with LLM-generated advice. LLMs ranked significantly higher overall and on effectiveness, warmth, and willingness to seek advice again. GPT-4o beat GPT-5 on all metrics except sycophancy, suggesting that benchmark gains need not improve advice-giving. In Study-2, we examined how human and algorithmic advice could be combined, and found that human advice can be unobtrusively polished to compete with AI-generated comments. We conclude with design implications for advice-giving agents and ecosystems blending AI, crowd input, and expert oversight. |
| title | When AI Gives Advice: Evaluating AI and Human Responses to Online Advice-Seeking for Well-Being |
| topic | Human-Computer Interaction Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2512.08937 |