Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.01917 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917183855001600 |
|---|---|
| author | Iwaki, Ryo Osogami, Takayuki |
| author_facet | Iwaki, Ryo Osogami, Takayuki |
| contents | While Distributional Reinforcement Learning (DRL) methods have demonstrated strong performance in online settings, its success in offline scenarios remains limited. We hypothesize that a key limitation of existing offline DRL methods lies in their approach to uniformly underestimate return quantiles. This uniform pessimism can lead to overly conservative value estimates, ultimately hindering generalization and performance. To address this, we introduce a novel concept called quantile distortion, which enables non-uniform pessimism by adjusting the degree of conservatism based on the availability of supporting data. Our approach is grounded in theoretical analysis and empirically validated, demonstrating improved performance over uniform pessimism. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_01917 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Distorted Distributional Policy Evaluation for Offline Reinforcement Learning Iwaki, Ryo Osogami, Takayuki Machine Learning While Distributional Reinforcement Learning (DRL) methods have demonstrated strong performance in online settings, its success in offline scenarios remains limited. We hypothesize that a key limitation of existing offline DRL methods lies in their approach to uniformly underestimate return quantiles. This uniform pessimism can lead to overly conservative value estimates, ultimately hindering generalization and performance. To address this, we introduce a novel concept called quantile distortion, which enables non-uniform pessimism by adjusting the degree of conservatism based on the availability of supporting data. Our approach is grounded in theoretical analysis and empirically validated, demonstrating improved performance over uniform pessimism. |
| title | Distorted Distributional Policy Evaluation for Offline Reinforcement Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2601.01917 |