Dynamic guessing for Hamiltonian Monte Carlo with embedded numerical root-finding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911011957637120 |
|---|---|
| author | Groves, Teddy Cowie, Nicholas Luke Nielsen, Lars Keld |
| author_facet | Groves, Teddy Cowie, Nicholas Luke Nielsen, Lars Keld |
| contents | Modern implementations of Hamiltonian Monte Carlo and related MCMC algorithms support sampling of probability functions that embed numerical root-finding algorithms, thereby allowing fitting of statistical models involving analytically intractable algebraic constraints. However the application of these models in practice is limited by the computational cost of computing large numbers of numerical solutions. We identify a key limitation of previous approaches to HMC with embedded root-finding, which require the starting guess to be the same at all points on the same simulated Hamiltonian trajectory. We demonstrate that this requirement can be relaxed, so that the starting guess depends on the previous integrator state. To choose a good guess using this information we propose two heuristics: use the previous solution and extrapolate the previous solution using implicit differentiation. Both heuristics yield substantial performance improvements on a range of representative models compared with static guessing. We also present grapevine, a JAX-based Python package providing easy access to an implementation of the No-U-Turn sampler augmented with dynamic guessing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15423 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic guessing for Hamiltonian Monte Carlo with embedded numerical root-finding Groves, Teddy Cowie, Nicholas Luke Nielsen, Lars Keld Computation Modern implementations of Hamiltonian Monte Carlo and related MCMC algorithms support sampling of probability functions that embed numerical root-finding algorithms, thereby allowing fitting of statistical models involving analytically intractable algebraic constraints. However the application of these models in practice is limited by the computational cost of computing large numbers of numerical solutions. We identify a key limitation of previous approaches to HMC with embedded root-finding, which require the starting guess to be the same at all points on the same simulated Hamiltonian trajectory. We demonstrate that this requirement can be relaxed, so that the starting guess depends on the previous integrator state. To choose a good guess using this information we propose two heuristics: use the previous solution and extrapolate the previous solution using implicit differentiation. Both heuristics yield substantial performance improvements on a range of representative models compared with static guessing. We also present grapevine, a JAX-based Python package providing easy access to an implementation of the No-U-Turn sampler augmented with dynamic guessing. |
| title | Dynamic guessing for Hamiltonian Monte Carlo with embedded numerical root-finding |
| topic | Computation |
| url | https://arxiv.org/abs/2506.15423 |