FlowCritic: Bridging Value Estimation with Flow Matching in Reinforcement Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915579144699904 |
|---|---|
| author | Zhong, Shan Ding, Shutong Diao, He Wang, Xiangyu Teh, Kah Chan Peng, Bei |
| author_facet | Zhong, Shan Ding, Shutong Diao, He Wang, Xiangyu Teh, Kah Chan Peng, Bei |
| contents | Reliable value estimation serves as the cornerstone of reinforcement learning (RL) by evaluating long-term returns and guiding policy improvement, significantly influencing the convergence speed and final performance. Existing works improve the reliability of value function estimation via multi-critic ensembles and distributional RL, yet the former merely combines multi point estimation without capturing distributional information, whereas the latter relies on discretization or quantile regression, limiting the expressiveness of complex value distributions. Inspired by flow matching's success in generative modeling, we propose a generative paradigm for value estimation, named FlowCritic. Departing from conventional regression for deterministic value prediction, FlowCritic leverages flow matching to model value distributions and generate samples for value estimation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_22686 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FlowCritic: Bridging Value Estimation with Flow Matching in Reinforcement Learning Zhong, Shan Ding, Shutong Diao, He Wang, Xiangyu Teh, Kah Chan Peng, Bei Machine Learning Artificial Intelligence Reliable value estimation serves as the cornerstone of reinforcement learning (RL) by evaluating long-term returns and guiding policy improvement, significantly influencing the convergence speed and final performance. Existing works improve the reliability of value function estimation via multi-critic ensembles and distributional RL, yet the former merely combines multi point estimation without capturing distributional information, whereas the latter relies on discretization or quantile regression, limiting the expressiveness of complex value distributions. Inspired by flow matching's success in generative modeling, we propose a generative paradigm for value estimation, named FlowCritic. Departing from conventional regression for deterministic value prediction, FlowCritic leverages flow matching to model value distributions and generate samples for value estimation. |
| title | FlowCritic: Bridging Value Estimation with Flow Matching in Reinforcement Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2510.22686 |