Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916934348439552 |
|---|---|
| author | Dutta, Anandi Mruthyunjaya, Shivani Saddington, Jessica Islam, Kazi Sifatul |
| author_facet | Dutta, Anandi Mruthyunjaya, Shivani Saddington, Jessica Islam, Kazi Sifatul |
| contents | The emergence of large language models (LLMs) has unlocked boundless possibilities, along with significant challenges. In response, we developed a mental health support chatbot designed to augment professional healthcare, with a strong emphasis on safe and meaningful application. Our approach involved rigorous evaluation, covering accuracy, empathy, trustworthiness, privacy, and bias. We employed a retrieval-augmented generation (RAG) framework, integrated prompt engineering, and fine-tuned a pre-trained model on novel datasets. The resulting system, Mentalic Net Conversational AI, achieved a BERT Score of 0.898, with other evaluation metrics falling within satisfactory ranges. We advocate for a human-in-the-loop approach and a long-term, responsible strategy in developing such transformative technologies, recognizing both their potential to change lives and the risks they may pose if not carefully managed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_04456 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support Dutta, Anandi Mruthyunjaya, Shivani Saddington, Jessica Islam, Kazi Sifatul Computation and Language The emergence of large language models (LLMs) has unlocked boundless possibilities, along with significant challenges. In response, we developed a mental health support chatbot designed to augment professional healthcare, with a strong emphasis on safe and meaningful application. Our approach involved rigorous evaluation, covering accuracy, empathy, trustworthiness, privacy, and bias. We employed a retrieval-augmented generation (RAG) framework, integrated prompt engineering, and fine-tuned a pre-trained model on novel datasets. The resulting system, Mentalic Net Conversational AI, achieved a BERT Score of 0.898, with other evaluation metrics falling within satisfactory ranges. We advocate for a human-in-the-loop approach and a long-term, responsible strategy in developing such transformative technologies, recognizing both their potential to change lives and the risks they may pose if not carefully managed. |
| title | Mentalic Net: Development of RAG-based Conversational AI and Evaluation Framework for Mental Health Support |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.04456 |