Multi-Objective Alignment of Language Models for Personalized Psychotherapy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908840337866752 |
|---|---|
| author | Beikzadeh, Mehrab Salmanpour, Yasaman Asadollah Suvarna, Ashima Sankararaman, Sriram Malgaroli, Matteo Sarrafzadeh, Majid Gabriel, Saadia |
| author_facet | Beikzadeh, Mehrab Salmanpour, Yasaman Asadollah Suvarna, Ashima Sankararaman, Sriram Malgaroli, Matteo Sarrafzadeh, Majid Gabriel, Saadia |
| contents | Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_16053 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multi-Objective Alignment of Language Models for Personalized Psychotherapy Beikzadeh, Mehrab Salmanpour, Yasaman Asadollah Suvarna, Ashima Sankararaman, Sriram Malgaroli, Matteo Sarrafzadeh, Majid Gabriel, Saadia Machine Learning Computation and Language Mental health disorders affect over 1 billion people worldwide, yet access to care remains limited by workforce shortages and cost constraints. While AI systems show therapeutic promise, current alignment approaches optimize objectives independently, failing to balance patient preferences with clinical safety. We survey 335 individuals with lived mental health experience to collect preference rankings across therapeutic dimensions, then develop a multi-objective alignment framework using direct preference optimization. We train reward models for six criteria -- empathy, safety, active listening, self-motivated change, trust/rapport, and patient autonomy -- and systematically compare multi-objective approaches against single-objective optimization, supervised fine-tuning, and parameter merging. Multi-objective DPO (MODPO) achieves superior balance (77.6% empathy, 62.6% safety) compared to single-objective optimization (93.6% empathy, 47.8% safety), and therapeutic criteria outperform general communication principles by 17.2%. Blinded clinician evaluation confirms MODPO is consistently preferred, with LLM-evaluator agreement comparable to inter-clinician reliability. |
| title | Multi-Objective Alignment of Language Models for Personalized Psychotherapy |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2602.16053 |