Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911416505597952 |
|---|---|
| author | Zhang, Guangyi Cai, Yunlong Yu, Guanding Simeone, Osvaldo |
| author_facet | Zhang, Guangyi Cai, Yunlong Yu, Guanding Simeone, Osvaldo |
| contents | We study the problem of monitoring model performance in dynamic environments where labeled data are limited. To this end, we propose prediction-powered risk monitoring (PPRM), a semi-supervised risk-monitoring approach based on prediction-powered inference (PPI). PPRM constructs anytime-valid lower bounds on the running risk by combining synthetic labels with a small set of true labels. Harmful shifts are detected via a threshold-based comparison with an upper bound on the nominal risk, satisfying assumption-free finite-sample guarantees in the probability of false alarm. We demonstrate the effectiveness of PPRM through extensive experiments on image classification, large language model (LLM), and telecommunications monitoring tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02229 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts Zhang, Guangyi Cai, Yunlong Yu, Guanding Simeone, Osvaldo Machine Learning Signal Processing We study the problem of monitoring model performance in dynamic environments where labeled data are limited. To this end, we propose prediction-powered risk monitoring (PPRM), a semi-supervised risk-monitoring approach based on prediction-powered inference (PPI). PPRM constructs anytime-valid lower bounds on the running risk by combining synthetic labels with a small set of true labels. Harmful shifts are detected via a threshold-based comparison with an upper bound on the nominal risk, satisfying assumption-free finite-sample guarantees in the probability of false alarm. We demonstrate the effectiveness of PPRM through extensive experiments on image classification, large language model (LLM), and telecommunications monitoring tasks. |
| title | Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts |
| topic | Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2602.02229 |