Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914001028382720 |
|---|---|
| author | Zhou, Zhijian An, Junyi Liu, Zongkai Shi, Yunfei Zhang, Xuan Cao, Fenglei Qu, Chao Qi, Yuan |
| author_facet | Zhou, Zhijian An, Junyi Liu, Zongkai Shi, Yunfei Zhang, Xuan Cao, Fenglei Qu, Chao Qi, Yuan |
| contents | Generating physically realistic 3D molecular structures remains a core challenge in molecular generative modeling. While diffusion models equipped with equivariant neural networks have made progress in capturing molecular geometries, they often struggle to produce equilibrium structures that adhere to physical principles such as force field consistency. To bridge this gap, we propose Reinforcement Learning with Physical Feedback (RLPF), a novel framework that extends Denoising Diffusion Policy Optimization to 3D molecular generation. RLPF formulates the task as a Markov decision process and applies proximal policy optimization to fine-tune equivariant diffusion models. Crucially, RLPF introduces reward functions derived from force-field evaluations, providing direct physical feedback to guide the generation toward energetically stable and physically meaningful structures. Experiments on the QM9 and GEOM-drug datasets demonstrate that RLPF significantly improves molecular stability compared to existing methods. These results highlight the value of incorporating physics-based feedback into generative modeling. The code is available at: https://github.com/ZhijianZhou/RLPF/tree/verl_diffusion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_16521 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation Zhou, Zhijian An, Junyi Liu, Zongkai Shi, Yunfei Zhang, Xuan Cao, Fenglei Qu, Chao Qi, Yuan Machine Learning Artificial Intelligence Generating physically realistic 3D molecular structures remains a core challenge in molecular generative modeling. While diffusion models equipped with equivariant neural networks have made progress in capturing molecular geometries, they often struggle to produce equilibrium structures that adhere to physical principles such as force field consistency. To bridge this gap, we propose Reinforcement Learning with Physical Feedback (RLPF), a novel framework that extends Denoising Diffusion Policy Optimization to 3D molecular generation. RLPF formulates the task as a Markov decision process and applies proximal policy optimization to fine-tune equivariant diffusion models. Crucially, RLPF introduces reward functions derived from force-field evaluations, providing direct physical feedback to guide the generation toward energetically stable and physically meaningful structures. Experiments on the QM9 and GEOM-drug datasets demonstrate that RLPF significantly improves molecular stability compared to existing methods. These results highlight the value of incorporating physics-based feedback into generative modeling. The code is available at: https://github.com/ZhijianZhou/RLPF/tree/verl_diffusion. |
| title | Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.16521 |