Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908384776683520 |
|---|---|
| author | Rabanser, Stephan Shamsabadi, Ali Shahin Franzese, Olive Wang, Xiao Weller, Adrian Papernot, Nicolas |
| author_facet | Rabanser, Stephan Shamsabadi, Ali Shahin Franzese, Olive Wang, Xiao Weller, Adrian Papernot, Nicolas |
| contents | Cautious predictions -- where a machine learning model abstains when uncertain -- are crucial for limiting harmful errors in safety-critical applications. In this work, we identify a novel threat: a dishonest institution can exploit these mechanisms to discriminate or unjustly deny services under the guise of uncertainty. We demonstrate the practicality of this threat by introducing an uncertainty-inducing attack called Mirage, which deliberately reduces confidence in targeted input regions, thereby covertly disadvantaging specific individuals. At the same time, Mirage maintains high predictive performance across all data points. To counter this threat, we propose Confidential Guardian, a framework that analyzes calibration metrics on a reference dataset to detect artificially suppressed confidence. Additionally, it employs zero-knowledge proofs of verified inference to ensure that reported confidence scores genuinely originate from the deployed model. This prevents the provider from fabricating arbitrary model confidence values while protecting the model's proprietary details. Our results confirm that Confidential Guardian effectively prevents the misuse of cautious predictions, providing verifiable assurances that abstention reflects genuine model uncertainty rather than malicious intent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23968 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention Rabanser, Stephan Shamsabadi, Ali Shahin Franzese, Olive Wang, Xiao Weller, Adrian Papernot, Nicolas Cryptography and Security Artificial Intelligence Computers and Society Machine Learning Cautious predictions -- where a machine learning model abstains when uncertain -- are crucial for limiting harmful errors in safety-critical applications. In this work, we identify a novel threat: a dishonest institution can exploit these mechanisms to discriminate or unjustly deny services under the guise of uncertainty. We demonstrate the practicality of this threat by introducing an uncertainty-inducing attack called Mirage, which deliberately reduces confidence in targeted input regions, thereby covertly disadvantaging specific individuals. At the same time, Mirage maintains high predictive performance across all data points. To counter this threat, we propose Confidential Guardian, a framework that analyzes calibration metrics on a reference dataset to detect artificially suppressed confidence. Additionally, it employs zero-knowledge proofs of verified inference to ensure that reported confidence scores genuinely originate from the deployed model. This prevents the provider from fabricating arbitrary model confidence values while protecting the model's proprietary details. Our results confirm that Confidential Guardian effectively prevents the misuse of cautious predictions, providing verifiable assurances that abstention reflects genuine model uncertainty rather than malicious intent. |
| title | Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention |
| topic | Cryptography and Security Artificial Intelligence Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2505.23968 |