FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914615571513344 |
|---|---|
| author | Kim, Sungha Lee, Gawon Lee, Jusuk Park, Jonghae Kim, H. Jin Cho, Daesol |
| author_facet | Kim, Sungha Lee, Gawon Lee, Jusuk Park, Jonghae Kim, H. Jin Cho, Daesol |
| contents | Maximum entropy reinforcement learning (MaxEnt-RL) enables robust exploration, yet practical implementations often restrict policies to simple Gaussians.
While recent approaches incorporate expressive generative policies via importance-weighted supervised learning, they are prone to importance weight collapse, which limits their scalability in high-dimensional action spaces.
Our key insight is to mitigate this limitation by localizing the sampling region, avoiding the weight degeneracy induced by importance sampling over the entire action space.
To instantiate this insight, we introduce \textbf{FLAG} (\textbf{F}low policy with \textbf{L}atent-\textbf{A}ugmented \textbf{G}uidance).
FLAG augments the state space with a flow latent variable and optimizes a provably consistent proxy MaxEnt-RL objective.
We empirically demonstrate that FLAG enables expressive policy optimization with limited importance samples and scales to high-dimensional control tasks.
Furthermore, FLAG achieves state-of-the-art performance across challenging benchmarks. Our project webpage: https://flag-rl.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30749 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance Kim, Sungha Lee, Gawon Lee, Jusuk Park, Jonghae Kim, H. Jin Cho, Daesol Machine Learning Robotics Maximum entropy reinforcement learning (MaxEnt-RL) enables robust exploration, yet practical implementations often restrict policies to simple Gaussians. While recent approaches incorporate expressive generative policies via importance-weighted supervised learning, they are prone to importance weight collapse, which limits their scalability in high-dimensional action spaces. Our key insight is to mitigate this limitation by localizing the sampling region, avoiding the weight degeneracy induced by importance sampling over the entire action space. To instantiate this insight, we introduce \textbf{FLAG} (\textbf{F}low policy with \textbf{L}atent-\textbf{A}ugmented \textbf{G}uidance). FLAG augments the state space with a flow latent variable and optimizes a provably consistent proxy MaxEnt-RL objective. We empirically demonstrate that FLAG enables expressive policy optimization with limited importance samples and scales to high-dimensional control tasks. Furthermore, FLAG achieves state-of-the-art performance across challenging benchmarks. Our project webpage: https://flag-rl.github.io/ |
| title | FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2605.30749 |