Boosting Stochastic Optimisation for High-dimensional Latent Variable Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915658217816064 |
|---|---|
| author | Oka, Motonori Chen, Yunxiao Moustaki, Irini |
| author_facet | Oka, Motonori Chen, Yunxiao Moustaki, Irini |
| contents | Latent variable models are widely used in social and behavioural sciences, including education, psychology, and political science. With the increasing availability of large and complex datasets, high-dimensional latent variable models have become more common. However, estimating such models via marginal maximum likelihood is computationally challenging because it requires evaluating a large number of high-dimensional integrals. Stochastic optimisation, which combines stochastic approximation and sampling techniques, has been shown to be effective. It iterates between sampling latent variables from their posterior distribution under current parameter estimates and updating the model parameters using an approximate stochastic gradient constructed from the latent variable samples. In this paper, we investigate strategies to improve the performance of stochastic optimisation for high-dimensional latent variable models. The improvement is achieved through two strategies: a Metropolis-adjusted Langevin sampler that uses the gradient of the negative complete-data log-likelihood to sample latent variables efficiently, and a minibatch gradient technique that uses only a subset of observations when sampling latent variables and constructing stochastic gradients. Our simulation studies show that combining these strategies yields the best overall performance among competitors. An application to a personality test with 30 latent dimensions further demonstrates that the proposed algorithm scales effectively to high-dimensional settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09311 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Boosting Stochastic Optimisation for High-dimensional Latent Variable Models Oka, Motonori Chen, Yunxiao Moustaki, Irini Computation Latent variable models are widely used in social and behavioural sciences, including education, psychology, and political science. With the increasing availability of large and complex datasets, high-dimensional latent variable models have become more common. However, estimating such models via marginal maximum likelihood is computationally challenging because it requires evaluating a large number of high-dimensional integrals. Stochastic optimisation, which combines stochastic approximation and sampling techniques, has been shown to be effective. It iterates between sampling latent variables from their posterior distribution under current parameter estimates and updating the model parameters using an approximate stochastic gradient constructed from the latent variable samples. In this paper, we investigate strategies to improve the performance of stochastic optimisation for high-dimensional latent variable models. The improvement is achieved through two strategies: a Metropolis-adjusted Langevin sampler that uses the gradient of the negative complete-data log-likelihood to sample latent variables efficiently, and a minibatch gradient technique that uses only a subset of observations when sampling latent variables and constructing stochastic gradients. Our simulation studies show that combining these strategies yields the best overall performance among competitors. An application to a personality test with 30 latent dimensions further demonstrates that the proposed algorithm scales effectively to high-dimensional settings. |
| title | Boosting Stochastic Optimisation for High-dimensional Latent Variable Models |
| topic | Computation |
| url | https://arxiv.org/abs/2406.09311 |