Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.14768 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908986236731392 |
|---|---|
| author | Hayakawa, Satoshi Morimura, Tetsuro |
| author_facet | Hayakawa, Satoshi Morimura, Tetsuro |
| contents | Reward evaluation of episodes becomes a bottleneck in a broad range of reinforcement learning tasks. Our aim in this paper is to select a small but representative subset of a large batch of episodes, only on which we actually compute rewards for more efficient policy gradient iterations. We build a Gaussian process modeling of discounted returns or rewards to derive a positive definite kernel on the space of episodes, run an ``episodic" kernel quadrature method to compress the information of sample episodes, and pass the reduced episodes to the policy network for gradient updates. We present the theoretical background of this procedure as well as its numerical illustrations in MuJoCo tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_14768 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Policy Gradient with Kernel Quadrature Hayakawa, Satoshi Morimura, Tetsuro Machine Learning Artificial Intelligence Reward evaluation of episodes becomes a bottleneck in a broad range of reinforcement learning tasks. Our aim in this paper is to select a small but representative subset of a large batch of episodes, only on which we actually compute rewards for more efficient policy gradient iterations. We build a Gaussian process modeling of discounted returns or rewards to derive a positive definite kernel on the space of episodes, run an ``episodic" kernel quadrature method to compress the information of sample episodes, and pass the reduced episodes to the policy network for gradient updates. We present the theoretical background of this procedure as well as its numerical illustrations in MuJoCo tasks. |
| title | Policy Gradient with Kernel Quadrature |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.14768 |