Optimal navigation in two-dimensional flows: control theory and reinforcement learning
Fuente:
arXiv
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912788883963904 |
|---|---|
| author | Parfenyev, Vladimir |
| author_facet | Parfenyev, Vladimir |
| contents | Zermelo's navigation problem seeks the trajectory of minimal travel time between two points in a fluid flow. We address this problem for an agent -- such as a micro-robot or active particle -- that is advected by a two-dimensional flow, self-propels at a fixed speed smaller than or comparable to the characteristic flow velocity, and can steer its direction. The flows considered span increasing levels of complexity, from steady solid-body rotation to the Taylor-Green flow and fully developed turbulence in the inverse cascade regime. Although optimal control theory provides time-minimizing trajectories, these solutions become unstable in chaotic regimes realized for complex background flows. To design robust navigation strategies under such conditions, we apply reinforcement learning. Both action-value (Q-learning) and policy-gradient (one-step actor-critic) methods achieve successful navigation with comparable performance. Crucially, we show that agents trained on coarse-grained turbulent flows -- retaining only large-scale features -- generalize effectively to the full velocity field. This robustness to incomplete flow information is essential for practical navigation in real-world oceanic and atmospheric environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08766 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimal navigation in two-dimensional flows: control theory and reinforcement learning Parfenyev, Vladimir Fluid Dynamics Zermelo's navigation problem seeks the trajectory of minimal travel time between two points in a fluid flow. We address this problem for an agent -- such as a micro-robot or active particle -- that is advected by a two-dimensional flow, self-propels at a fixed speed smaller than or comparable to the characteristic flow velocity, and can steer its direction. The flows considered span increasing levels of complexity, from steady solid-body rotation to the Taylor-Green flow and fully developed turbulence in the inverse cascade regime. Although optimal control theory provides time-minimizing trajectories, these solutions become unstable in chaotic regimes realized for complex background flows. To design robust navigation strategies under such conditions, we apply reinforcement learning. Both action-value (Q-learning) and policy-gradient (one-step actor-critic) methods achieve successful navigation with comparable performance. Crucially, we show that agents trained on coarse-grained turbulent flows -- retaining only large-scale features -- generalize effectively to the full velocity field. This robustness to incomplete flow information is essential for practical navigation in real-world oceanic and atmospheric environments. |
| title | Optimal navigation in two-dimensional flows: control theory and reinforcement learning |
| topic | Fluid Dynamics |
| url | https://arxiv.org/abs/2512.08766 |