Strategy Complexity of Limsup and Liminf Threshold Objectives in Countable MDPs, with Applications to Optimal Expected Payoffs
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917778711117824 |
|---|---|
| author | Mayr, Richard Munday, Eric |
| author_facet | Mayr, Richard Munday, Eric |
| contents | We study Markov decision processes (MDPs) with a countably infinite number of states. The $\limsup$ (resp. $\liminf$) threshold objective is to maximize the probability that the $\limsup$ (resp. $\liminf$) of the infinite sequence of directly seen rewards is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., the upper and lower bounds on the memory required by $\varepsilon$-optimal (resp. optimal) strategies. We then apply these results to solve two open problems from (Sudderth, Decisions in Economics and Finance, 2020) about the strategy complexity of optimal strategies for the expected $\limsup$ (resp. $\liminf$) payoff. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_13259 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Strategy Complexity of Limsup and Liminf Threshold Objectives in Countable MDPs, with Applications to Optimal Expected Payoffs Mayr, Richard Munday, Eric Optimization and Control Probability 90C40, 91A60 We study Markov decision processes (MDPs) with a countably infinite number of states. The $\limsup$ (resp. $\liminf$) threshold objective is to maximize the probability that the $\limsup$ (resp. $\liminf$) of the infinite sequence of directly seen rewards is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., the upper and lower bounds on the memory required by $\varepsilon$-optimal (resp. optimal) strategies. We then apply these results to solve two open problems from (Sudderth, Decisions in Economics and Finance, 2020) about the strategy complexity of optimal strategies for the expected $\limsup$ (resp. $\liminf$) payoff. |
| title | Strategy Complexity of Limsup and Liminf Threshold Objectives in Countable MDPs, with Applications to Optimal Expected Payoffs |
| topic | Optimization and Control Probability 90C40, 91A60 |
| url | https://arxiv.org/abs/2211.13259 |