Predictive Churn with the Set of Good Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916707723902976 |
|---|---|
| author | Watson-Daniels, Jamelle Calmon, Flavio du Pin D'Amour, Alexander Long, Carol Parkes, David C. Ustun, Berk |
| author_facet | Watson-Daniels, Jamelle Calmon, Flavio du Pin D'Amour, Alexander Long, Carol Parkes, David C. Ustun, Berk |
| contents | Issues can arise when research focused on fairness, transparency, or safety is conducted separately from research driven by practical deployment concerns and vice versa. This separation creates a growing need for translational work that bridges the gap between independently studied concepts that may be fundamentally related. This paper explores connections between two seemingly unrelated concepts of predictive inconsistency that share intriguing parallels. The first, known as predictive multiplicity, occurs when models that perform similarly (e.g., nearly equivalent training loss) produce conflicting predictions for individual samples. This concept is often emphasized in algorithmic fairness research as a means of promoting transparency in ML model development. The second concept, predictive churn, examines the differences in individual predictions before and after model updates, a key challenge in deploying ML models in consumer-facing applications. We present theoretical and empirical results that uncover links between these previously disconnected concepts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_07745 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Predictive Churn with the Set of Good Models Watson-Daniels, Jamelle Calmon, Flavio du Pin D'Amour, Alexander Long, Carol Parkes, David C. Ustun, Berk Machine Learning Issues can arise when research focused on fairness, transparency, or safety is conducted separately from research driven by practical deployment concerns and vice versa. This separation creates a growing need for translational work that bridges the gap between independently studied concepts that may be fundamentally related. This paper explores connections between two seemingly unrelated concepts of predictive inconsistency that share intriguing parallels. The first, known as predictive multiplicity, occurs when models that perform similarly (e.g., nearly equivalent training loss) produce conflicting predictions for individual samples. This concept is often emphasized in algorithmic fairness research as a means of promoting transparency in ML model development. The second concept, predictive churn, examines the differences in individual predictions before and after model updates, a key challenge in deploying ML models in consumer-facing applications. We present theoretical and empirical results that uncover links between these previously disconnected concepts. |
| title | Predictive Churn with the Set of Good Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.07745 |