Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917624476073984 |
|---|---|
| author | Kumar, Harsh Li, Tong Shi, Jiakai Musabirov, Ilya Kornfield, Rachel Meyerhoff, Jonah Bhattacharjee, Ananya Karr, Chris Nguyen, Theresa Mohr, David Rafferty, Anna Villar, Sofia Deliu, Nina Williams, Joseph Jay |
| author_facet | Kumar, Harsh Li, Tong Shi, Jiakai Musabirov, Ilya Kornfield, Rachel Meyerhoff, Jonah Bhattacharjee, Ananya Karr, Chris Nguyen, Theresa Mohr, David Rafferty, Anna Villar, Sofia Deliu, Nina Williams, Joseph Jay |
| contents | Digital mental health (DMH) interventions, such as text-message-based lessons and activities, offer immense potential for accessible mental health support. While these interventions can be effective, real-world experimental testing can further enhance their design and impact. Adaptive experimentation, utilizing algorithms like Thompson Sampling for (contextual) multi-armed bandit (MAB) problems, can lead to continuous improvement and personalization. However, it remains unclear when these algorithms can simultaneously increase user experience rewards and facilitate appropriate data collection for social-behavioral scientists to analyze with sufficient statistical confidence. Although a growing body of research addresses the practical and statistical aspects of MAB and other adaptive algorithms, further exploration is needed to assess their impact across diverse real-world contexts. This paper presents a software system developed over two years that allows text-messaging intervention components to be adapted using bandit and other algorithms while collecting data for side-by-side comparison with traditional uniform random non-adaptive experiments. We evaluate the system by deploying a text-message-based DMH intervention to 1100 users, recruited through a large mental health non-profit organization, and share the path forward for deploying this system at scale. This system not only enables applications in mental health but could also serve as a model testbed for adaptive experimentation algorithms in other domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_18326 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health Kumar, Harsh Li, Tong Shi, Jiakai Musabirov, Ilya Kornfield, Rachel Meyerhoff, Jonah Bhattacharjee, Ananya Karr, Chris Nguyen, Theresa Mohr, David Rafferty, Anna Villar, Sofia Deliu, Nina Williams, Joseph Jay Artificial Intelligence Computers and Society Human-Computer Interaction Machine Learning Digital mental health (DMH) interventions, such as text-message-based lessons and activities, offer immense potential for accessible mental health support. While these interventions can be effective, real-world experimental testing can further enhance their design and impact. Adaptive experimentation, utilizing algorithms like Thompson Sampling for (contextual) multi-armed bandit (MAB) problems, can lead to continuous improvement and personalization. However, it remains unclear when these algorithms can simultaneously increase user experience rewards and facilitate appropriate data collection for social-behavioral scientists to analyze with sufficient statistical confidence. Although a growing body of research addresses the practical and statistical aspects of MAB and other adaptive algorithms, further exploration is needed to assess their impact across diverse real-world contexts. This paper presents a software system developed over two years that allows text-messaging intervention components to be adapted using bandit and other algorithms while collecting data for side-by-side comparison with traditional uniform random non-adaptive experiments. We evaluate the system by deploying a text-message-based DMH intervention to 1100 users, recruited through a large mental health non-profit organization, and share the path forward for deploying this system at scale. This system not only enables applications in mental health but could also serve as a model testbed for adaptive experimentation algorithms in other domains. |
| title | Using Adaptive Bandit Experiments to Increase and Investigate Engagement in Mental Health |
| topic | Artificial Intelligence Computers and Society Human-Computer Interaction Machine Learning |
| url | https://arxiv.org/abs/2310.18326 |