Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2512.08936 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908702973362176 |
|---|---|
| author | Winslow, Brent Shreibati, Jacqueline Perez, Javier Su, Hao-Wei Young-Lin, Nichole Hammerquist, Nova McDuff, Daniel Guss, Jason Vafeiadou, Jenny Cain, Nick Lin, Alex Schenck, Erik Rajagopal, Shiva Chung, Jia-Ru Venkatakrishnan, Anusha Lee, Amy Armento Karimzadehgan, Maryam Meng, Qingyou Agarwal, Rythm Natarajan, Aravind Giest, Tracy |
| author_facet | Winslow, Brent Shreibati, Jacqueline Perez, Javier Su, Hao-Wei Young-Lin, Nichole Hammerquist, Nova McDuff, Daniel Guss, Jason Vafeiadou, Jenny Cain, Nick Lin, Alex Schenck, Erik Rajagopal, Shiva Chung, Jia-Ru Venkatakrishnan, Anusha Lee, Amy Armento Karimzadehgan, Maryam Meng, Qingyou Agarwal, Rythm Natarajan, Aravind Giest, Tracy |
| contents | The incorporation of generative artificial intelligence into personal health applications presents a transformative opportunity for personalized, data-driven health and fitness guidance, yet also poses challenges related to user safety, model accuracy, and personal privacy. To address these challenges, a novel, principle-based framework was developed and validated for the systematic evaluation of LLMs applied to personal health and wellness. First, the development of the Fitbit Insights explorer, a large language model (LLM)-powered system designed to help users interpret their personal health data, is described. Subsequently, the safety, helpfulness, accuracy, relevance, and personalization (SHARP) principle-based framework is introduced as an end-to-end operational methodology that integrates comprehensive evaluation techniques including human evaluation by generalists and clinical specialists, autorater assessments, and adversarial testing, into an iterative development lifecycle. Through the application of this framework to the Fitbit Insights explorer in a staged deployment involving over 13,000 consented users, challenges not apparent during initial testing were systematically identified. This process guided targeted improvements to the system and demonstrated the necessity of combining isolated technical evaluations with real-world user feedback. Finally, a comprehensive, actionable approach is established for the responsible development and deployment of LLM-powered health applications, providing a standardized methodology to foster innovation while ensuring emerging technologies are safe, effective, and trustworthy for users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08936 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness Winslow, Brent Shreibati, Jacqueline Perez, Javier Su, Hao-Wei Young-Lin, Nichole Hammerquist, Nova McDuff, Daniel Guss, Jason Vafeiadou, Jenny Cain, Nick Lin, Alex Schenck, Erik Rajagopal, Shiva Chung, Jia-Ru Venkatakrishnan, Anusha Lee, Amy Armento Karimzadehgan, Maryam Meng, Qingyou Agarwal, Rythm Natarajan, Aravind Giest, Tracy Human-Computer Interaction Artificial Intelligence Computers and Society The incorporation of generative artificial intelligence into personal health applications presents a transformative opportunity for personalized, data-driven health and fitness guidance, yet also poses challenges related to user safety, model accuracy, and personal privacy. To address these challenges, a novel, principle-based framework was developed and validated for the systematic evaluation of LLMs applied to personal health and wellness. First, the development of the Fitbit Insights explorer, a large language model (LLM)-powered system designed to help users interpret their personal health data, is described. Subsequently, the safety, helpfulness, accuracy, relevance, and personalization (SHARP) principle-based framework is introduced as an end-to-end operational methodology that integrates comprehensive evaluation techniques including human evaluation by generalists and clinical specialists, autorater assessments, and adversarial testing, into an iterative development lifecycle. Through the application of this framework to the Fitbit Insights explorer in a staged deployment involving over 13,000 consented users, challenges not apparent during initial testing were systematically identified. This process guided targeted improvements to the system and demonstrated the necessity of combining isolated technical evaluations with real-world user feedback. Finally, a comprehensive, actionable approach is established for the responsible development and deployment of LLM-powered health applications, providing a standardized methodology to foster innovation while ensuring emerging technologies are safe, effective, and trustworthy for users. |
| title | A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness |
| topic | Human-Computer Interaction Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2512.08936 |