Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2403.07495 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913262304493568 |
|---|---|
| author | Tran, Jimmy Huy Kleppe, Tore Selland |
| author_facet | Tran, Jimmy Huy Kleppe, Tore Selland |
| contents | Three approaches for adaptively tuning diagonal scale matrices for HMC are discussed and compared. The common practice of scaling according to estimated marginal standard deviations is taken as a benchmark. Scaling according to the mean log-target gradient (ISG), and a scaling method targeting that the frequency of when the underlying Hamiltonian dynamics crosses the respective medians should be uniform across dimensions, are taken as alternatives. Numerical studies suggest that the ISG method leads in many cases to more efficient sampling than the benchmark, in particular in cases with strong correlations or non-linear dependencies. The ISG method is also easy to implement, computationally cheap and would be relatively simple to include in automatically tuned codes as an alternative to the benchmark practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07495 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Tuning diagonal scale matrices for HMC Tran, Jimmy Huy Kleppe, Tore Selland Computation Methodology Machine Learning Three approaches for adaptively tuning diagonal scale matrices for HMC are discussed and compared. The common practice of scaling according to estimated marginal standard deviations is taken as a benchmark. Scaling according to the mean log-target gradient (ISG), and a scaling method targeting that the frequency of when the underlying Hamiltonian dynamics crosses the respective medians should be uniform across dimensions, are taken as alternatives. Numerical studies suggest that the ISG method leads in many cases to more efficient sampling than the benchmark, in particular in cases with strong correlations or non-linear dependencies. The ISG method is also easy to implement, computationally cheap and would be relatively simple to include in automatically tuned codes as an alternative to the benchmark practice. |
| title | Tuning diagonal scale matrices for HMC |
| topic | Computation Methodology Machine Learning |
| url | https://arxiv.org/abs/2403.07495 |