RuAG: Learned-rule-augmented Generation for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912106807296000 |
|---|---|
| author | Zhang, Yudi Xiao, Pei Wang, Lu Zhang, Chaoyun Fang, Meng Du, Yali Puzyrev, Yevgeniy Yao, Randolph Qin, Si Lin, Qingwei Pechenizkiy, Mykola Zhang, Dongmei Rajmohan, Saravan Zhang, Qi |
| author_facet | Zhang, Yudi Xiao, Pei Wang, Lu Zhang, Chaoyun Fang, Meng Du, Yali Puzyrev, Yevgeniy Yao, Randolph Qin, Si Lin, Qingwei Pechenizkiy, Mykola Zhang, Dongmei Rajmohan, Saravan Zhang, Qi |
| contents | In-context learning (ICL) and Retrieval-Augmented Generation (RAG) have gained attention for their ability to enhance LLMs' reasoning by incorporating external knowledge but suffer from limited contextual window size, leading to insufficient information injection. To this end, we propose a novel framework, RuAG, to automatically distill large volumes of offline data into interpretable first-order logic rules, which are injected into LLMs to boost their reasoning capabilities. Our method begins by formulating the search process relying on LLMs' commonsense, where LLMs automatically define head and body predicates. Then, RuAG applies Monte Carlo Tree Search (MCTS) to address the combinational searching space and efficiently discover logic rules from data. The resulting logic rules are translated into natural language, allowing targeted knowledge injection and seamless integration into LLM prompts for LLM's downstream task reasoning. We evaluate our framework on public and private industrial tasks, including natural language processing, time-series, decision-making, and industrial tasks, demonstrating its effectiveness in enhancing LLM's capability over diverse tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_03349 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RuAG: Learned-rule-augmented Generation for Large Language Models Zhang, Yudi Xiao, Pei Wang, Lu Zhang, Chaoyun Fang, Meng Du, Yali Puzyrev, Yevgeniy Yao, Randolph Qin, Si Lin, Qingwei Pechenizkiy, Mykola Zhang, Dongmei Rajmohan, Saravan Zhang, Qi Artificial Intelligence Computation and Language Machine Learning In-context learning (ICL) and Retrieval-Augmented Generation (RAG) have gained attention for their ability to enhance LLMs' reasoning by incorporating external knowledge but suffer from limited contextual window size, leading to insufficient information injection. To this end, we propose a novel framework, RuAG, to automatically distill large volumes of offline data into interpretable first-order logic rules, which are injected into LLMs to boost their reasoning capabilities. Our method begins by formulating the search process relying on LLMs' commonsense, where LLMs automatically define head and body predicates. Then, RuAG applies Monte Carlo Tree Search (MCTS) to address the combinational searching space and efficiently discover logic rules from data. The resulting logic rules are translated into natural language, allowing targeted knowledge injection and seamless integration into LLM prompts for LLM's downstream task reasoning. We evaluate our framework on public and private industrial tasks, including natural language processing, time-series, decision-making, and industrial tasks, demonstrating its effectiveness in enhancing LLM's capability over diverse tasks. |
| title | RuAG: Learned-rule-augmented Generation for Large Language Models |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2411.03349 |