Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2604.23760 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911623416905728 |
|---|---|
| author | Polatov, Yishay Sabag, Oron |
| author_facet | Polatov, Yishay Sabag, Oron |
| contents | We study the control of finite-state systems driven by exogenous disturbances, and design causal policies that track the performance of a lookahead benchmark controller. This objective is formalized through dynamic regret, so that favorable disturbance sequences are compared against a strong benchmark, while under adverse disturbance sequences the comparison accounts for the benchmark's degraded performance. This benchmark-relative framework provides an alternative to classical MDP formulations, which assume i.i.d. disturbances, and to robust control approaches, which optimize against worst-case disturbances. Our main result is a nested dynamic-programming solution that computes both the optimal worst-case regret and a regret-optimal policy. In particular, we introduce the Regret-Bellman operator, whose fixed-point value function feeds into a finite-horizon dynamic program. Numerical examples show that regret-optimal policies interpolate nicely between MDP-based and robust controllers without requiring knowledge of the disturbance distribution, and can even outperform both under i.i.d. or structured disturbances. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_23760 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Regret-Optimal Control for Finite-State Systems Polatov, Yishay Sabag, Oron Optimization and Control We study the control of finite-state systems driven by exogenous disturbances, and design causal policies that track the performance of a lookahead benchmark controller. This objective is formalized through dynamic regret, so that favorable disturbance sequences are compared against a strong benchmark, while under adverse disturbance sequences the comparison accounts for the benchmark's degraded performance. This benchmark-relative framework provides an alternative to classical MDP formulations, which assume i.i.d. disturbances, and to robust control approaches, which optimize against worst-case disturbances. Our main result is a nested dynamic-programming solution that computes both the optimal worst-case regret and a regret-optimal policy. In particular, we introduce the Regret-Bellman operator, whose fixed-point value function feeds into a finite-horizon dynamic program. Numerical examples show that regret-optimal policies interpolate nicely between MDP-based and robust controllers without requiring knowledge of the disturbance distribution, and can even outperform both under i.i.d. or structured disturbances. |
| title | Regret-Optimal Control for Finite-State Systems |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2604.23760 |