PAGER: A Framework for Failure Analysis of Deep Regression Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914818379743232 |
|---|---|
| author | Thiagarajan, Jayaraman J. Narayanaswamy, Vivek Trivedi, Puja Anirudh, Rushil |
| author_facet | Thiagarajan, Jayaraman J. Narayanaswamy, Vivek Trivedi, Puja Anirudh, Rushil |
| contents | Safe deployment of AI models requires proactive detection of failures to prevent costly errors. To this end, we study the important problem of detecting failures in deep regression models. Existing approaches rely on epistemic uncertainty estimates or inconsistency w.r.t the training data to identify failure. Interestingly, we find that while uncertainties are necessary they are insufficient to accurately characterize failure in practice. Hence, we introduce PAGER (Principled Analysis of Generalization Errors in Regressors), a framework to systematically detect and characterize failures in deep regressors. Built upon the principle of anchored training in deep models, PAGER unifies both epistemic uncertainty and complementary manifold non-conformity scores to accurately organize samples into different risk regimes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_10977 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | PAGER: A Framework for Failure Analysis of Deep Regression Models Thiagarajan, Jayaraman J. Narayanaswamy, Vivek Trivedi, Puja Anirudh, Rushil Machine Learning Safe deployment of AI models requires proactive detection of failures to prevent costly errors. To this end, we study the important problem of detecting failures in deep regression models. Existing approaches rely on epistemic uncertainty estimates or inconsistency w.r.t the training data to identify failure. Interestingly, we find that while uncertainties are necessary they are insufficient to accurately characterize failure in practice. Hence, we introduce PAGER (Principled Analysis of Generalization Errors in Regressors), a framework to systematically detect and characterize failures in deep regressors. Built upon the principle of anchored training in deep models, PAGER unifies both epistemic uncertainty and complementary manifold non-conformity scores to accurately organize samples into different risk regimes. |
| title | PAGER: A Framework for Failure Analysis of Deep Regression Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2309.10977 |