Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.19429 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915947443388416 |
|---|---|
| author | Shulman, Yefim Kitkowska, Agnieszka Warner, Mark |
| author_facet | Shulman, Yefim Kitkowska, Agnieszka Warner, Mark |
| contents | For online health communities, community trust is paramount. Yet, advances in Large Language Models (LLMs) generating advice may erode this trust, especially if users cannot identify whether LLMs have been used. We investigate the feasibility of community-based detection of health advice authorship and how self-moderation of LLMs could help enhance advice utilization. In an online experiment, we evaluate people's ability to distinguish AI-generated from human-written advice across two health conditions, considering lived experience with a condition, AI-recognition training, and user attitudes towards transparency and trust around AI use. Our results indicate the need for transparency coupled with trust. We find little evidence of people's ability to discern advice authorship. However, we find a consistent effect of the health condition. Our qualitative findings identify unreliable signals, resulting in flawed heuristic evaluations of the advice. Our findings point to opportunities to improve the self-moderation of LLM-based AI and aid community-based AI moderation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19429 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Discerning Authorship in Online Health Communities: Experience, Trust, and Transparency Implications for Moderating AI Shulman, Yefim Kitkowska, Agnieszka Warner, Mark Human-Computer Interaction Computers and Society For online health communities, community trust is paramount. Yet, advances in Large Language Models (LLMs) generating advice may erode this trust, especially if users cannot identify whether LLMs have been used. We investigate the feasibility of community-based detection of health advice authorship and how self-moderation of LLMs could help enhance advice utilization. In an online experiment, we evaluate people's ability to distinguish AI-generated from human-written advice across two health conditions, considering lived experience with a condition, AI-recognition training, and user attitudes towards transparency and trust around AI use. Our results indicate the need for transparency coupled with trust. We find little evidence of people's ability to discern advice authorship. However, we find a consistent effect of the health condition. Our qualitative findings identify unreliable signals, resulting in flawed heuristic evaluations of the advice. Our findings point to opportunities to improve the self-moderation of LLM-based AI and aid community-based AI moderation. |
| title | Discerning Authorship in Online Health Communities: Experience, Trust, and Transparency Implications for Moderating AI |
| topic | Human-Computer Interaction Computers and Society |
| url | https://arxiv.org/abs/2604.19429 |