Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915459391029248 |
|---|---|
| author | Qu, Yanlin Blanchet, Jose Glynn, Peter |
| author_facet | Qu, Yanlin Blanchet, Jose Glynn, Peter |
| contents | Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson's equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16737 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions Qu, Yanlin Blanchet, Jose Glynn, Peter Machine Learning Probability Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson's equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond. |
| title | Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions |
| topic | Machine Learning Probability |
| url | https://arxiv.org/abs/2508.16737 |