Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910678837624832 |
|---|---|
| author | Vakili, Sattar Olkhovskaya, Julia |
| author_facet | Vakili, Sattar Olkhovskaya, Julia |
| contents | Reinforcement learning utilizing kernel ridge regression to predict the expected value function represents a powerful method with great representational capacity. This setting is a highly versatile framework amenable to analytical results. We consider kernel-based function approximation for RL in the infinite horizon average reward setting, also referred to as the undiscounted setting. We propose an optimistic algorithm, similar to acquisition function based algorithms in the special case of bandits. We establish novel no-regret performance guarantees for our algorithm, under kernel-based modelling assumptions. Additionally, we derive a novel confidence interval for the kernel-based prediction of the expected value function, applicable across various RL problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_23498 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm Vakili, Sattar Olkhovskaya, Julia Machine Learning Artificial Intelligence Reinforcement learning utilizing kernel ridge regression to predict the expected value function represents a powerful method with great representational capacity. This setting is a highly versatile framework amenable to analytical results. We consider kernel-based function approximation for RL in the infinite horizon average reward setting, also referred to as the undiscounted setting. We propose an optimistic algorithm, similar to acquisition function based algorithms in the special case of bandits. We establish novel no-regret performance guarantees for our algorithm, under kernel-based modelling assumptions. Additionally, we derive a novel confidence interval for the kernel-based prediction of the expected value function, applicable across various RL problems. |
| title | Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithm |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.23498 |