Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.14972 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917276755689472 |
|---|---|
| author | Reuter, Arik Dhir, Anish Diaconu, Cristiana Robertson, Jake Ossen, Ole Hutter, Frank Weller, Adrian van der Wilk, Mark Schölkopf, Bernhard |
| author_facet | Reuter, Arik Dhir, Anish Diaconu, Cristiana Robertson, Jake Ossen, Ole Hutter, Frank Weller, Adrian van der Wilk, Mark Schölkopf, Bernhard |
| contents | Estimating causal quantities traditionally relies on bespoke estimators tailored to specific assumptions. Recently proposed Causal Foundation Models (CFMs) promise a more unified approach by amortising causal discovery and inference in a single step. However, in their current state, they do not allow for the incorporation of any domain knowledge, which can lead to suboptimal predictions. We bridge this gap by introducing methods to condition CFMs on causal information, such as the causal graph or more readily available ancestral information. When access to complete causal graph information is too strict a requirement, our approach also effectively leverages partial causal information. We systematically evaluate conditioning strategies and find that injecting learnable biases into the attention mechanism is the most effective method to utilise full and partial causal information. Our experiments show that this conditioning allows a general-purpose CFM to match the performance of specialised models trained on specific causal structures. Overall, our approach addresses a central hurdle on the path towards all-in-one causal foundation models: the capability to answer causal queries in a data-driven manner while effectively leveraging any amount of domain expertise. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_14972 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Use What You Know: Causal Foundation Models with Partial Graphs Reuter, Arik Dhir, Anish Diaconu, Cristiana Robertson, Jake Ossen, Ole Hutter, Frank Weller, Adrian van der Wilk, Mark Schölkopf, Bernhard Machine Learning Estimating causal quantities traditionally relies on bespoke estimators tailored to specific assumptions. Recently proposed Causal Foundation Models (CFMs) promise a more unified approach by amortising causal discovery and inference in a single step. However, in their current state, they do not allow for the incorporation of any domain knowledge, which can lead to suboptimal predictions. We bridge this gap by introducing methods to condition CFMs on causal information, such as the causal graph or more readily available ancestral information. When access to complete causal graph information is too strict a requirement, our approach also effectively leverages partial causal information. We systematically evaluate conditioning strategies and find that injecting learnable biases into the attention mechanism is the most effective method to utilise full and partial causal information. Our experiments show that this conditioning allows a general-purpose CFM to match the performance of specialised models trained on specific causal structures. Overall, our approach addresses a central hurdle on the path towards all-in-one causal foundation models: the capability to answer causal queries in a data-driven manner while effectively leveraging any amount of domain expertise. |
| title | Use What You Know: Causal Foundation Models with Partial Graphs |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.14972 |