Gradient Estimation with Discrete Stein Operators
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929312650756096 |
|---|---|
| author | Shi, Jiaxin Zhou, Yuhao Hwang, Jessica Titsias, Michalis K. Mackey, Lester |
| author_facet | Shi, Jiaxin Zhou, Yuhao Hwang, Jessica Titsias, Michalis K. Mackey, Lester |
| contents | Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2202_09497 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Gradient Estimation with Discrete Stein Operators Shi, Jiaxin Zhou, Yuhao Hwang, Jessica Titsias, Michalis K. Mackey, Lester Machine Learning Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations. |
| title | Gradient Estimation with Discrete Stein Operators |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2202.09497 |