PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917187830153216 |
|---|---|
| author | Shen, Yaling Fong, Stephanie Jiang, Yiwen Wang, Zimu Tang, Feilong Xu, Qingyang Zhao, Xiangyu Xu, Zhongxing Liu, Jiahe Hu, Jinpeng Dwyer, Dominic Ge, Zongyuan |
| author_facet | Shen, Yaling Fong, Stephanie Jiang, Yiwen Wang, Zimu Tang, Feilong Xu, Qingyang Zhao, Xiangyu Xu, Zhongxing Liu, Jiahe Hu, Jinpeng Dwyer, Dominic Ge, Zongyuan |
| contents | The increasing integration of large language models (LLMs) into mental health applications necessitates robust frameworks for evaluating professional safety alignment. Current evaluative approaches primarily rely on refusal-based safety signals, which offer limited insight into the nuanced behaviors required in clinical practice. In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking. To address this gap, we move beyond refusal-centric metrics and introduce \texttt{PsychEthicsBench}, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs' ethical knowledge and behavioral responses through multiple-choice and open-ended tasks with fine-grained ethicality annotations. Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness. Notably, we find that domain-specific fine-tuning can degrade ethical robustness, as several specialized models underperform their base backbones in ethical alignment. PsychEthicsBench provides a foundation for systematic, jurisdiction-aware evaluation of LLMs in mental health, encouraging more responsible development in this domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03578 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics Shen, Yaling Fong, Stephanie Jiang, Yiwen Wang, Zimu Tang, Feilong Xu, Qingyang Zhao, Xiangyu Xu, Zhongxing Liu, Jiahe Hu, Jinpeng Dwyer, Dominic Ge, Zongyuan Computation and Language The increasing integration of large language models (LLMs) into mental health applications necessitates robust frameworks for evaluating professional safety alignment. Current evaluative approaches primarily rely on refusal-based safety signals, which offer limited insight into the nuanced behaviors required in clinical practice. In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking. To address this gap, we move beyond refusal-centric metrics and introduce \texttt{PsychEthicsBench}, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs' ethical knowledge and behavioral responses through multiple-choice and open-ended tasks with fine-grained ethicality annotations. Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness. Notably, we find that domain-specific fine-tuning can degrade ethical robustness, as several specialized models underperform their base backbones in ethical alignment. PsychEthicsBench provides a foundation for systematic, jurisdiction-aware evaluation of LLMs in mental health, encouraging more responsible development in this domain. |
| title | PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.03578 |