Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.14925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917845161476096 |
|---|---|
| author | Lu, Linqi Deng, Yifan Tian, Chuan Yang, Sijia Shah, Dhavan |
| author_facet | Lu, Linqi Deng, Yifan Tian, Chuan Yang, Sijia Shah, Dhavan |
| contents | This study introduces Purrfessor, an innovative AI chatbot designed to provide personalized dietary guidance through interactive, multimodal engagement. Leveraging the Large Language-and-Vision Assistant (LLaVA) model fine-tuned with food and nutrition data and a human-in-the-loop approach, Purrfessor integrates visual meal analysis with contextual advice to enhance user experience and engagement. We conducted two studies to evaluate the chatbot's performance and user experience: (a) simulation assessments and human validation were conducted to examine the performance of the fine-tuned model; (b) a 2 (Profile: Bot vs. Pet) by 3 (Model: GPT-4 vs. LLaVA vs. Fine-tuned LLaVA) experiment revealed that Purrfessor significantly enhanced users' perceptions of care ($β= 1.59$, $p = 0.04$) and interest ($β= 2.26$, $p = 0.01$) compared to the GPT-4 bot. Additionally, user interviews highlighted the importance of interaction design details, emphasizing the need for responsiveness, personalization, and guidance to improve user engagement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_14925 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot Lu, Linqi Deng, Yifan Tian, Chuan Yang, Sijia Shah, Dhavan Human-Computer Interaction Artificial Intelligence This study introduces Purrfessor, an innovative AI chatbot designed to provide personalized dietary guidance through interactive, multimodal engagement. Leveraging the Large Language-and-Vision Assistant (LLaVA) model fine-tuned with food and nutrition data and a human-in-the-loop approach, Purrfessor integrates visual meal analysis with contextual advice to enhance user experience and engagement. We conducted two studies to evaluate the chatbot's performance and user experience: (a) simulation assessments and human validation were conducted to examine the performance of the fine-tuned model; (b) a 2 (Profile: Bot vs. Pet) by 3 (Model: GPT-4 vs. LLaVA vs. Fine-tuned LLaVA) experiment revealed that Purrfessor significantly enhanced users' perceptions of care ($β= 1.59$, $p = 0.04$) and interest ($β= 2.26$, $p = 0.01$) compared to the GPT-4 bot. Additionally, user interviews highlighted the importance of interaction design details, emphasizing the need for responsiveness, personalization, and guidance to improve user engagement. |
| title | Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2411.14925 |