From "Help" to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915810742632448 |
|---|---|
| author | Steigerwald, Philipp Albrecht, Jens |
| author_facet | Steigerwald, Philipp Albrecht, Jens |
| contents | Psychosocial online counselling frequently encounters generic subject lines that impede efficient case prioritisation. This study evaluates eleven large language models generating six-word subject lines for German counselling emails through hierarchical assessment - first categorising outputs, then ranking within categories to enable manageable evaluation. Nine assessors (counselling professionals and AI systems) enable analysis via Krippendorff's $α$, Spearman's $ρ$, Pearson's $r$ and Kendall's $τ$. Results reveal performance trade-offs between proprietary services and privacy-preserving open-source alternatives, with German fine-tuning consistently improving performance. The study addresses critical ethical considerations for mental health AI deployment including privacy, bias and accountability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_18443 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | From "Help" to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications Steigerwald, Philipp Albrecht, Jens Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society Psychosocial online counselling frequently encounters generic subject lines that impede efficient case prioritisation. This study evaluates eleven large language models generating six-word subject lines for German counselling emails through hierarchical assessment - first categorising outputs, then ranking within categories to enable manageable evaluation. Nine assessors (counselling professionals and AI systems) enable analysis via Krippendorff's $α$, Spearman's $ρ$, Pearson's $r$ and Kendall's $τ$. Results reveal performance trade-offs between proprietary services and privacy-preserving open-source alternatives, with German fine-tuning consistently improving performance. The study addresses critical ethical considerations for mental health AI deployment including privacy, bias and accountability. |
| title | From "Help" to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications |
| topic | Human-Computer Interaction Artificial Intelligence Computation and Language Computers and Society |
| url | https://arxiv.org/abs/2602.18443 |