Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.19434 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912085619769344 |
|---|---|
| author | Hoxha, Isabelle Sperber, Leo Palminteri, Stefano |
| author_facet | Hoxha, Isabelle Sperber, Leo Palminteri, Stefano |
| contents | The tendency of repeating past choices more often than expected from the history of outcomes has been repeatedly empirically observed in reinforcement learning experiments. It can be explained by at least two computational processes: asymmetric update and (gradual) choice perseveration. A recent meta-analysis showed that both mechanisms are detectable in human reinforcement learning. However, while their descriptive value seems to be well established, they have not been compared regarding their possible adaptive value. In this study, we address this gap by simulating reinforcement learning agents in a variety of environments with a new variant of an evolutionary algorithm. Our results show that positivity bias (in the form of asymmetric update) is evolutionary stable in many situations, while the emergence of gradual perseveration is less systematic and robust. Overall, our results illustrate that biases can be adaptive and selected by evolution, in an environment-specific manner. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_19434 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evolving choice hysteresis in reinforcement learning: comparing the adaptive value of positivity bias and gradual perseveration Hoxha, Isabelle Sperber, Leo Palminteri, Stefano Neural and Evolutionary Computing Neurons and Cognition The tendency of repeating past choices more often than expected from the history of outcomes has been repeatedly empirically observed in reinforcement learning experiments. It can be explained by at least two computational processes: asymmetric update and (gradual) choice perseveration. A recent meta-analysis showed that both mechanisms are detectable in human reinforcement learning. However, while their descriptive value seems to be well established, they have not been compared regarding their possible adaptive value. In this study, we address this gap by simulating reinforcement learning agents in a variety of environments with a new variant of an evolutionary algorithm. Our results show that positivity bias (in the form of asymmetric update) is evolutionary stable in many situations, while the emergence of gradual perseveration is less systematic and robust. Overall, our results illustrate that biases can be adaptive and selected by evolution, in an environment-specific manner. |
| title | Evolving choice hysteresis in reinforcement learning: comparing the adaptive value of positivity bias and gradual perseveration |
| topic | Neural and Evolutionary Computing Neurons and Cognition |
| url | https://arxiv.org/abs/2410.19434 |