Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911735132192768 |
|---|---|
| author | Kim, Kwangho Kim, Jisu |
| author_facet | Kim, Kwangho Kim, Jisu |
| contents | In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce. We study semi-supervised regression in this setting, and propose a simple two stage estimator that learns kernel eigenfeatures from all proxy covariates and fits a ridge predictor on labeled data. We derive finite sample bounds showing that fast labeled sample rates are recovered when proxy perturbation is controlled and unlabeled proxy covariates are sufficiently abundant. We also show that distribution regression is a direct special case, with analogous guarantees when the finite bag size is large enough. Experiments show consistent gains over supervised and semi-supervised baselines, especially in low label regimes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_00512 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression Kim, Kwangho Kim, Jisu Machine Learning Information Theory In many modern machine learning pipelines, abundant pretrained representations serve as noisy proxy covariates, while task-specific labels remain scarce. We study semi-supervised regression in this setting, and propose a simple two stage estimator that learns kernel eigenfeatures from all proxy covariates and fits a ridge predictor on labeled data. We derive finite sample bounds showing that fast labeled sample rates are recovered when proxy perturbation is controlled and unlabeled proxy covariates are sufficiently abundant. We also show that distribution regression is a direct special case, with analogous guarantees when the finite bag size is large enough. Experiments show consistent gains over supervised and semi-supervised baselines, especially in low label regimes. |
| title | Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression |
| topic | Machine Learning Information Theory |
| url | https://arxiv.org/abs/2606.00512 |