Task Scarcity and Label Leakage in Relational Transfer Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912992579289088 |
|---|---|
| author | Azevedo, Francisco Galuppo Loures, Clarissa Lima Correa, Denis Oliveira |
| author_facet | Azevedo, Francisco Galuppo Loures, Clarissa Lima Correa, Denis Oliveira |
| contents | Training relational foundation models requires learning representations that transfer across tasks, yet available supervision is typically limited to a small number of prediction targets per database. This task scarcity causes learned representations to encode task-specific shortcuts that degrade transfer even within the same schema, a problem we call label leakage. We study this using K-Space, a modular architecture combining frozen pretrained tabular encoders with a lightweight message-passing core. To suppress leakage, we introduce a gradient projection method that removes label-predictive directions from representation updates. On RelBench, this improves within-dataset transfer by +0.145 AUROC on average, often recovering near single-task performance. Our results suggest that limited task diversity, not just limited data, constrains relational foundation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_29914 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Task Scarcity and Label Leakage in Relational Transfer Learning Azevedo, Francisco Galuppo Loures, Clarissa Lima Correa, Denis Oliveira Machine Learning Training relational foundation models requires learning representations that transfer across tasks, yet available supervision is typically limited to a small number of prediction targets per database. This task scarcity causes learned representations to encode task-specific shortcuts that degrade transfer even within the same schema, a problem we call label leakage. We study this using K-Space, a modular architecture combining frozen pretrained tabular encoders with a lightweight message-passing core. To suppress leakage, we introduce a gradient projection method that removes label-predictive directions from representation updates. On RelBench, this improves within-dataset transfer by +0.145 AUROC on average, often recovering near single-task performance. Our results suggest that limited task diversity, not just limited data, constrains relational foundation models. |
| title | Task Scarcity and Label Leakage in Relational Transfer Learning |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2603.29914 |