An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916733846028288 |
|---|---|
| author | Hua, Mengjian Laurière, Mathieu Vanden-Eijnden, Eric |
| author_facet | Hua, Mengjian Laurière, Mathieu Vanden-Eijnden, Eric |
| contents | We present a novel on-policy algorithm for solving stochastic optimal control (SOC) problems. By leveraging the Girsanov theorem, our method directly computes on-policy gradients of the SOC objective without expensive backpropagation through stochastic differential equations or adjoint problem solutions. This approach significantly accelerates the optimization of neural network control policies while scaling efficiently to high-dimensional problems and long time horizons. We evaluate our method on classical SOC benchmarks as well as applications to sampling from unnormalized distributions via Schrödinger-Föllmer processes and fine-tuning pre-trained diffusion models. Experimental results demonstrate substantial improvements in both computational speed and memory efficiency compared to existing approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_05163 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control Hua, Mengjian Laurière, Mathieu Vanden-Eijnden, Eric Machine Learning Optimization and Control We present a novel on-policy algorithm for solving stochastic optimal control (SOC) problems. By leveraging the Girsanov theorem, our method directly computes on-policy gradients of the SOC objective without expensive backpropagation through stochastic differential equations or adjoint problem solutions. This approach significantly accelerates the optimization of neural network control policies while scaling efficiently to high-dimensional problems and long time horizons. We evaluate our method on classical SOC benchmarks as well as applications to sampling from unnormalized distributions via Schrödinger-Föllmer processes and fine-tuning pre-trained diffusion models. Experimental results demonstrate substantial improvements in both computational speed and memory efficiency compared to existing approaches. |
| title | An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2410.05163 |