RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909566665490432 |
|---|---|
| author | Dai, Weichen Dai, Zijie Huang, Zhijie Pan, Yixuan Li, Xinhe Li, Xi Zhou, Yi Qi, Ji Jiang, Wu |
| author_facet | Dai, Weichen Dai, Zijie Huang, Zhijie Pan, Yixuan Li, Xinhe Li, Xi Zhou, Yi Qi, Ji Jiang, Wu |
| contents | While current large language models (LLMs) demonstrate remarkable linguistic capabilities through training on massive unstructured text corpora, they remain inadequate in leveraging structured scientific data (e.g., chemical molecular properties in databases) that encapsulate centuries of accumulated scientific expertise. These structured datasets hold strategic significance for advancing AI for Science yet current approaches merely treat them as auxiliary supplements to unstructured text. This study pioneers a systematic investigation into enhancing LLMs with structured scientific data, using chemical molecular science as a testbed. We investigate the impact of incorporating molecular property data on LLM across distinct training phases, including continual pre-training, supervised fine-tuning, and reinforcement learning. Notably, to address the inherent limitation of numerical insensitivity in large models, we propose an innovative methodology termed "Reinforcement Learning with Database Feedback" (RLDBF). Experimental evaluations demonstrate the efficacy of the proposed approach, with the model exhibiting remarkable generalization capabilities on previously unseen data and other chemical tasks. The results substantiate the potential of our method in advancing the field of structured scientific data processing within LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_03713 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack Dai, Weichen Dai, Zijie Huang, Zhijie Pan, Yixuan Li, Xinhe Li, Xi Zhou, Yi Qi, Ji Jiang, Wu Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science While current large language models (LLMs) demonstrate remarkable linguistic capabilities through training on massive unstructured text corpora, they remain inadequate in leveraging structured scientific data (e.g., chemical molecular properties in databases) that encapsulate centuries of accumulated scientific expertise. These structured datasets hold strategic significance for advancing AI for Science yet current approaches merely treat them as auxiliary supplements to unstructured text. This study pioneers a systematic investigation into enhancing LLMs with structured scientific data, using chemical molecular science as a testbed. We investigate the impact of incorporating molecular property data on LLM across distinct training phases, including continual pre-training, supervised fine-tuning, and reinforcement learning. Notably, to address the inherent limitation of numerical insensitivity in large models, we propose an innovative methodology termed "Reinforcement Learning with Database Feedback" (RLDBF). Experimental evaluations demonstrate the efficacy of the proposed approach, with the model exhibiting remarkable generalization capabilities on previously unseen data and other chemical tasks. The results substantiate the potential of our method in advancing the field of structured scientific data processing within LLMs. |
| title | RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack |
| topic | Machine Learning Artificial Intelligence Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2504.03713 |