Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.00265 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909769455894528 |
|---|---|
| author | Ouattara, Koffi Ismael Krontiris, Ioannis Dimitrakos, Theo Kargl, Frank |
| author_facet | Ouattara, Koffi Ismael Krontiris, Ioannis Dimitrakos, Theo Kargl, Frank |
| contents | Trustworthiness in neural networks is crucial for their deployment in critical applications, where reliability, confidence, and uncertainty play pivotal roles in decision-making. Traditional performance metrics such as accuracy and precision fail to capture these aspects, particularly in cases where models exhibit overconfidence. To address these limitations, this paper introduces a novel framework for quantifying the trustworthiness of neural networks by incorporating subjective logic into the evaluation of Expected Calibration Error (ECE). This method provides a comprehensive measure of trust, disbelief, and uncertainty by clustering predicted probabilities and fusing opinions using appropriate fusion operators. We demonstrate the effectiveness of this approach through experiments on MNIST and CIFAR-10 datasets, where post-calibration results indicate improved trustworthiness. The proposed framework offers a more interpretable and nuanced assessment of AI models, with potential applications in sensitive domains such as healthcare and autonomous systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_00265 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Quantifying Calibration Error in Neural Networks Through Evidence-Based Theory Ouattara, Koffi Ismael Krontiris, Ioannis Dimitrakos, Theo Kargl, Frank Machine Learning Artificial Intelligence Logic Trustworthiness in neural networks is crucial for their deployment in critical applications, where reliability, confidence, and uncertainty play pivotal roles in decision-making. Traditional performance metrics such as accuracy and precision fail to capture these aspects, particularly in cases where models exhibit overconfidence. To address these limitations, this paper introduces a novel framework for quantifying the trustworthiness of neural networks by incorporating subjective logic into the evaluation of Expected Calibration Error (ECE). This method provides a comprehensive measure of trust, disbelief, and uncertainty by clustering predicted probabilities and fusing opinions using appropriate fusion operators. We demonstrate the effectiveness of this approach through experiments on MNIST and CIFAR-10 datasets, where post-calibration results indicate improved trustworthiness. The proposed framework offers a more interpretable and nuanced assessment of AI models, with potential applications in sensitive domains such as healthcare and autonomous systems. |
| title | Quantifying Calibration Error in Neural Networks Through Evidence-Based Theory |
| topic | Machine Learning Artificial Intelligence Logic |
| url | https://arxiv.org/abs/2411.00265 |