Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910813894213632 |
|---|---|
| author | Agrawal, Atharva Mangeshkumar Shinde, Rutika Pandurang Bhukya, Vasanth Kumar Chakraborty, Ashmita Shah, Sagar Bharat Shukla, Tanmay Relangi, Sree Pradeep Kumar Mutyam, Nilesh |
| author_facet | Agrawal, Atharva Mangeshkumar Shinde, Rutika Pandurang Bhukya, Vasanth Kumar Chakraborty, Ashmita Shah, Sagar Bharat Shukla, Tanmay Relangi, Sree Pradeep Kumar Mutyam, Nilesh |
| contents | Large language models (LLMs) have shown impressive capabilities in natural language processing tasks, including dialogue generation. This research aims to conduct a novel comparative analysis of two prominent techniques, fine-tuning with LoRA (Low-Rank Adaptation) and the Retrieval-Augmented Generation (RAG) framework, in the context of doctor-patient chat conversations with multiple datasets of mixed medical domains. The analysis involves three state-of-the-art models: Llama-2, GPT, and the LSTM model. Employing real-world doctor-patient dialogues, we comprehensively evaluate the performance of models, assessing key metrics such as language quality (perplexity, BLEU score), factual accuracy (fact-checking against medical knowledge bases), adherence to medical guidelines, and overall human judgments (coherence, empathy, safety). The findings provide insights into the strengths and limitations of each approach, shedding light on their suitability for healthcare applications. Furthermore, the research investigates the robustness of the models in handling diverse patient queries, ranging from general health inquiries to specific medical conditions. The impact of domain-specific knowledge integration is also explored, highlighting the potential for enhancing LLM performance through targeted data augmentation and retrieval strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02249 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation Agrawal, Atharva Mangeshkumar Shinde, Rutika Pandurang Bhukya, Vasanth Kumar Chakraborty, Ashmita Shah, Sagar Bharat Shukla, Tanmay Relangi, Sree Pradeep Kumar Mutyam, Nilesh Computation and Language Artificial Intelligence Large language models (LLMs) have shown impressive capabilities in natural language processing tasks, including dialogue generation. This research aims to conduct a novel comparative analysis of two prominent techniques, fine-tuning with LoRA (Low-Rank Adaptation) and the Retrieval-Augmented Generation (RAG) framework, in the context of doctor-patient chat conversations with multiple datasets of mixed medical domains. The analysis involves three state-of-the-art models: Llama-2, GPT, and the LSTM model. Employing real-world doctor-patient dialogues, we comprehensively evaluate the performance of models, assessing key metrics such as language quality (perplexity, BLEU score), factual accuracy (fact-checking against medical knowledge bases), adherence to medical guidelines, and overall human judgments (coherence, empathy, safety). The findings provide insights into the strengths and limitations of each approach, shedding light on their suitability for healthcare applications. Furthermore, the research investigates the robustness of the models in handling diverse patient queries, ranging from general health inquiries to specific medical conditions. The impact of domain-specific knowledge integration is also explored, highlighting the potential for enhancing LLM performance through targeted data augmentation and retrieval strategies. |
| title | Conversation AI Dialog for Medicare powered by Finetuning and Retrieval Augmented Generation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2502.02249 |