Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2301.13565 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908344780849152 |
|---|---|
| author | Wang, Shixiong Wang, Haowei Li, Xinke Honorio, Jean |
| author_facet | Wang, Shixiong Wang, Haowei Li, Xinke Honorio, Jean |
| contents | Trustworthy machine learning aims at combating distributional uncertainties in training data distributions compared to population distributions. Typical treatment frameworks include the Bayesian approach, (min-max) distributionally robust optimization (DRO), and regularization. However, three issues have to be raised: 1) the prior distribution in the Bayesian method and the regularizer in the regularization method are difficult to specify; 2) the DRO method tends to be overly conservative; 3) all the three methods are biased estimators of the true optimal cost. This paper studies a new framework that unifies the three approaches and addresses the three challenges above. The asymptotic properties (e.g., consistencies and asymptotic normalities), non-asymptotic properties (e.g., generalization bounds and unbiasedness), and solution methods of the proposed model are studied. The new model reveals the trade-off between the robustness to the unseen data and the specificity to the training data. Experiments on various real-world tasks validate the superiority of the proposed learning framework. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_13565 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity Wang, Shixiong Wang, Haowei Li, Xinke Honorio, Jean Machine Learning 62F35, 62G35, 62C12 Trustworthy machine learning aims at combating distributional uncertainties in training data distributions compared to population distributions. Typical treatment frameworks include the Bayesian approach, (min-max) distributionally robust optimization (DRO), and regularization. However, three issues have to be raised: 1) the prior distribution in the Bayesian method and the regularizer in the regularization method are difficult to specify; 2) the DRO method tends to be overly conservative; 3) all the three methods are biased estimators of the true optimal cost. This paper studies a new framework that unifies the three approaches and addresses the three challenges above. The asymptotic properties (e.g., consistencies and asymptotic normalities), non-asymptotic properties (e.g., generalization bounds and unbiasedness), and solution methods of the proposed model are studied. The new model reveals the trade-off between the robustness to the unseen data and the specificity to the training data. Experiments on various real-world tasks validate the superiority of the proposed learning framework. |
| title | Learning Against Distributional Uncertainty: On the Trade-off Between Robustness and Specificity |
| topic | Machine Learning 62F35, 62G35, 62C12 |
| url | https://arxiv.org/abs/2301.13565 |