Improving GFlowNets with Monte Carlo Tree Search
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916295434305536 |
|---|---|
| author | Morozov, Nikita Tiapkin, Daniil Samsonov, Sergey Naumov, Alexey Vetrov, Dmitry |
| author_facet | Morozov, Nikita Tiapkin, Daniil Samsonov, Sergey Naumov, Alexey Vetrov, Dmitry |
| contents | Generative Flow Networks (GFlowNets) treat sampling from distributions over compositional discrete spaces as a sequential decision-making problem, training a stochastic policy to construct objects step by step. Recent studies have revealed strong connections between GFlowNets and entropy-regularized reinforcement learning. Building on these insights, we propose to enhance planning capabilities of GFlowNets by applying Monte Carlo Tree Search (MCTS). Specifically, we show how the MENTS algorithm (Xiao et al., 2019) can be adapted for GFlowNets and used during both training and inference. Our experiments demonstrate that this approach improves the sample efficiency of GFlowNet training and the generation fidelity of pre-trained GFlowNet models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13655 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving GFlowNets with Monte Carlo Tree Search Morozov, Nikita Tiapkin, Daniil Samsonov, Sergey Naumov, Alexey Vetrov, Dmitry Machine Learning Artificial Intelligence Generative Flow Networks (GFlowNets) treat sampling from distributions over compositional discrete spaces as a sequential decision-making problem, training a stochastic policy to construct objects step by step. Recent studies have revealed strong connections between GFlowNets and entropy-regularized reinforcement learning. Building on these insights, we propose to enhance planning capabilities of GFlowNets by applying Monte Carlo Tree Search (MCTS). Specifically, we show how the MENTS algorithm (Xiao et al., 2019) can be adapted for GFlowNets and used during both training and inference. Our experiments demonstrate that this approach improves the sample efficiency of GFlowNet training and the generation fidelity of pre-trained GFlowNet models. |
| title | Improving GFlowNets with Monte Carlo Tree Search |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2406.13655 |