Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917500433727488 |
|---|---|
| author | Wu, Jingcheng Thapa, Ratan Bahadur Nayyeri, Mojtaba Etteldorf, Lucas Finkenbeiner, Max Leeske, Fabian Staab, Steffen |
| author_facet | Wu, Jingcheng Thapa, Ratan Bahadur Nayyeri, Mojtaba Etteldorf, Lucas Finkenbeiner, Max Leeske, Fabian Staab, Steffen |
| contents | Relational databases store much of the world's structured information, and they are essential for driving complex predictive applications. However, deep learning progress on relational data remains limited, as conventional approaches flatten databases into single tables via manual feature engineering, discarding relational context. Relational deep learning (RDL) addresses this by modeling databases as relational entity graphs (REGs) for graph neural networks (GNNs), but remains task- and database-specific. To combine the strengths of both paradigms, we propose a hybrid architecture combining a fine-tuned BART encoder to capture intra-row semantics with a GraphSAGE-based GNN over REGs to inject relational context. Experiments on RelBench show that the GNN substantially enriches BART's row embeddings, achieving a ROC-AUC of 67.40 on the driver-dnf task from the rel-f1 dataset. This performance is competitive with supervised baselines such as LightGBM (68.86) and narrows the gap to RDL (72.62) to within 5.22 points, though a substantial gap remains to state-of-the-art foundation models such as KumoRFM (82.63). These results suggest that lightweight hybrid LM-GNN architectures offer a promising and resource-efficient path towards foundation models for relational databases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16085 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks Wu, Jingcheng Thapa, Ratan Bahadur Nayyeri, Mojtaba Etteldorf, Lucas Finkenbeiner, Max Leeske, Fabian Staab, Steffen Databases Artificial Intelligence cs.AI Relational databases store much of the world's structured information, and they are essential for driving complex predictive applications. However, deep learning progress on relational data remains limited, as conventional approaches flatten databases into single tables via manual feature engineering, discarding relational context. Relational deep learning (RDL) addresses this by modeling databases as relational entity graphs (REGs) for graph neural networks (GNNs), but remains task- and database-specific. To combine the strengths of both paradigms, we propose a hybrid architecture combining a fine-tuned BART encoder to capture intra-row semantics with a GraphSAGE-based GNN over REGs to inject relational context. Experiments on RelBench show that the GNN substantially enriches BART's row embeddings, achieving a ROC-AUC of 67.40 on the driver-dnf task from the rel-f1 dataset. This performance is competitive with supervised baselines such as LightGBM (68.86) and narrows the gap to RDL (72.62) to within 5.22 points, though a substantial gap remains to state-of-the-art foundation models such as KumoRFM (82.63). These results suggest that lightweight hybrid LM-GNN architectures offer a promising and resource-efficient path towards foundation models for relational databases. |
| title | Towards Foundation Models for Relational Databases with Language Models and Graph Neural Networks |
| topic | Databases Artificial Intelligence cs.AI |
| url | https://arxiv.org/abs/2605.16085 |