Fitting Reinforcement Learning Model to Behavioral Data under Bandits
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908915336216576 |
|---|---|
| author | Zhu, Hao Hoffmann, Jasper Zhang, Baohe Boedecker, Joschka |
| author_facet | Zhu, Hao Hoffmann, Jasper Zhang, Baohe Boedecker, Joschka |
| contents | We consider the problem of fitting a reinforcement learning (RL) model to some given behavioral data under a multi-armed bandit environment. These models have received much attention in recent years for characterizing human and animal decision making behavior. We provide a generic mathematical optimization problem formulation for the fitting problem of a wide range of RL models that appear frequently in scientific research applications. We then provide a detailed theoretical analysis of its convexity properties. Based on the theoretical results, we introduce a novel solution method for the fitting problem of RL models based on convex relaxation and optimization. Our method is then evaluated in several simulated and real-world bandit environments to compare with some benchmark methods that appear in the literature. Numerical results indicate that our method achieves comparable performance to the state-of-the-art, while significantly reducing computation time. We also provide an open-source Python package for our proposed method to empower researchers to apply it in the analysis of their datasets directly, without prior knowledge of convex optimization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04454 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fitting Reinforcement Learning Model to Behavioral Data under Bandits Zhu, Hao Hoffmann, Jasper Zhang, Baohe Boedecker, Joschka Computational Engineering, Finance, and Science Machine Learning Optimization and Control Neurons and Cognition 90C25, 90C59, 90C90 We consider the problem of fitting a reinforcement learning (RL) model to some given behavioral data under a multi-armed bandit environment. These models have received much attention in recent years for characterizing human and animal decision making behavior. We provide a generic mathematical optimization problem formulation for the fitting problem of a wide range of RL models that appear frequently in scientific research applications. We then provide a detailed theoretical analysis of its convexity properties. Based on the theoretical results, we introduce a novel solution method for the fitting problem of RL models based on convex relaxation and optimization. Our method is then evaluated in several simulated and real-world bandit environments to compare with some benchmark methods that appear in the literature. Numerical results indicate that our method achieves comparable performance to the state-of-the-art, while significantly reducing computation time. We also provide an open-source Python package for our proposed method to empower researchers to apply it in the analysis of their datasets directly, without prior knowledge of convex optimization. |
| title | Fitting Reinforcement Learning Model to Behavioral Data under Bandits |
| topic | Computational Engineering, Finance, and Science Machine Learning Optimization and Control Neurons and Cognition 90C25, 90C59, 90C90 |
| url | https://arxiv.org/abs/2511.04454 |