Robust SGLD algorithm for solving non-convex distributionally robust optimisation problems
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913097577398272 |
|---|---|
| author | Neufeld, Ariel En, Matthew Ng Cheng Zhang, Ying |
| author_facet | Neufeld, Ariel En, Matthew Ng Cheng Zhang, Ying |
| contents | In this paper we develop a Stochastic Gradient Langevin Dynamics (SGLD) algorithm tailored for solving a certain class of non-convex distributionally robust optimisation (DRO) problems. By deriving non-asymptotic convergence bounds, we build an algorithm which for any prescribed accuracy $\varepsilon>0$ outputs an estimator whose expected excess risk is at most $\varepsilon$. As a concrete application, we consider the problem of identifying the best non-linear estimator of a given regression model involving a neural network using adversarially corrupted samples. We formulate this problem as a DRO problem and demonstrate both theoretically and numerically the applicability of the proposed robust SGLD algorithm. Moreover, numerical experiments show that the robust SGLD estimator outperforms the estimator obtained using vanilla SGLD in terms of test accuracy, which highlights the advantage of incorporating model uncertainty when optimising with perturbed samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09532 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Robust SGLD algorithm for solving non-convex distributionally robust optimisation problems Neufeld, Ariel En, Matthew Ng Cheng Zhang, Ying Optimization and Control Probability Mathematical Finance In this paper we develop a Stochastic Gradient Langevin Dynamics (SGLD) algorithm tailored for solving a certain class of non-convex distributionally robust optimisation (DRO) problems. By deriving non-asymptotic convergence bounds, we build an algorithm which for any prescribed accuracy $\varepsilon>0$ outputs an estimator whose expected excess risk is at most $\varepsilon$. As a concrete application, we consider the problem of identifying the best non-linear estimator of a given regression model involving a neural network using adversarially corrupted samples. We formulate this problem as a DRO problem and demonstrate both theoretically and numerically the applicability of the proposed robust SGLD algorithm. Moreover, numerical experiments show that the robust SGLD estimator outperforms the estimator obtained using vanilla SGLD in terms of test accuracy, which highlights the advantage of incorporating model uncertainty when optimising with perturbed samples. |
| title | Robust SGLD algorithm for solving non-convex distributionally robust optimisation problems |
| topic | Optimization and Control Probability Mathematical Finance |
| url | https://arxiv.org/abs/2403.09532 |