Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909976141758464 |
|---|---|
| author | Sharma, Aditya Gupta, Ananya Wang, Chengyu Adebayo, Chiamaka Kowalski, Jakub |
| author_facet | Sharma, Aditya Gupta, Ananya Wang, Chengyu Adebayo, Chiamaka Kowalski, Jakub |
| contents | Large Language Models (LLMs), despite their advanced linguistic capabilities, fundamentally lack an intuitive understanding of physical dynamics, which limits their effectiveness in real-world scenarios that require causal reasoning. In this paper, we introduce Causal World Model Induction (CWMI), a novel framework designed to embed an explicit model of causal physics within an LLM. Our approach incorporates a dedicated Causal Physics Module (CPM) and a new training objective called Causal Intervention Loss, encouraging the model to learn cause-and-effect relationships from multimodal data. By training the model to predict the outcomes of hypothetical interventions instead of merely capturing statistical correlations, CWMI develops a robust internal representation of physical laws. Experimental results show that CWMI significantly outperforms state-of-the-art LLMs on zero-shot physical reasoning tasks, including the PIQA benchmark and our newly proposed PhysiCa-Bench dataset. These findings demonstrate that inducing a causal world model is a critical step toward more reliable and generalizable AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_19855 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning Sharma, Aditya Gupta, Ananya Wang, Chengyu Adebayo, Chiamaka Kowalski, Jakub Machine Learning Human-Computer Interaction 68T05, 68T07, 68T40 I.2.6; I.2.9; I.2.7; I.2.10; H.5.2 Large Language Models (LLMs), despite their advanced linguistic capabilities, fundamentally lack an intuitive understanding of physical dynamics, which limits their effectiveness in real-world scenarios that require causal reasoning. In this paper, we introduce Causal World Model Induction (CWMI), a novel framework designed to embed an explicit model of causal physics within an LLM. Our approach incorporates a dedicated Causal Physics Module (CPM) and a new training objective called Causal Intervention Loss, encouraging the model to learn cause-and-effect relationships from multimodal data. By training the model to predict the outcomes of hypothetical interventions instead of merely capturing statistical correlations, CWMI develops a robust internal representation of physical laws. Experimental results show that CWMI significantly outperforms state-of-the-art LLMs on zero-shot physical reasoning tasks, including the PIQA benchmark and our newly proposed PhysiCa-Bench dataset. These findings demonstrate that inducing a causal world model is a critical step toward more reliable and generalizable AI systems. |
| title | Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning |
| topic | Machine Learning Human-Computer Interaction 68T05, 68T07, 68T40 I.2.6; I.2.9; I.2.7; I.2.10; H.5.2 |
| url | https://arxiv.org/abs/2507.19855 |