Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918514949881856 |
|---|---|
| author | Cai, Hengrui Liu, Shengjie Song, Rui |
| author_facet | Cai, Hengrui Liu, Shengjie Song, Rui |
| contents | This paper introduces a causal attribution model to enhance the interpretability of large language models (LLMs) and improve their causal reasoning abilities via precise fine-tuning. Despite LLMs' proficiency in diverse tasks, their reasoning processes often remain black box, and thus restrict targeted enhancement. We propose a novel causal attribution model that utilizes "do-operators" for constructing interventional scenarios, allowing us to quantify the contribution of different components in LLMs's causal reasoning process systematically. By assessing the proposed attribution scores through causal discovery tasks across various domains, we demonstrate that LLMs' effectiveness in causal discovery heavily relies on provided context and domain-specific knowledge but can also utilize numerical data with limited calculations in correlation, not causation. This motivates the proposed fine-tuned LLM for pairwise causal discovery, effectively and correctly leveraging both knowledge and numerical information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_00139 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning Cai, Hengrui Liu, Shengjie Song, Rui Artificial Intelligence Computation and Language Machine Learning Methodology This paper introduces a causal attribution model to enhance the interpretability of large language models (LLMs) and improve their causal reasoning abilities via precise fine-tuning. Despite LLMs' proficiency in diverse tasks, their reasoning processes often remain black box, and thus restrict targeted enhancement. We propose a novel causal attribution model that utilizes "do-operators" for constructing interventional scenarios, allowing us to quantify the contribution of different components in LLMs's causal reasoning process systematically. By assessing the proposed attribution scores through causal discovery tasks across various domains, we demonstrate that LLMs' effectiveness in causal discovery heavily relies on provided context and domain-specific knowledge but can also utilize numerical data with limited calculations in correlation, not causation. This motivates the proposed fine-tuned LLM for pairwise causal discovery, effectively and correctly leveraging both knowledge and numerical information. |
| title | Enhancing Causal Reasoning in Large Language Models: A Causal Attribution Model for Precision Fine-Tuning |
| topic | Artificial Intelligence Computation and Language Machine Learning Methodology |
| url | https://arxiv.org/abs/2401.00139 |