Phase transition for conditional covariance matrices estimated by importance sampling, and implications for cross-entropy schemes in high dimension
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914158592655360 |
|---|---|
| author | Beh, Jason Morio, Jerome Simatos, Florian |
| author_facet | Beh, Jason Morio, Jerome Simatos, Florian |
| contents | Motivated by the estimation of covariance matrices by importance sampling arising in the cross-entropy (CE) algorithm, we study a random matrix model $\hat Σ= {\bf X} L {\bf X}^\top$ with two distinct features: $\bf X$ and $L$ are dependent, and $L$ is heavy-tailed. In the high-dimensional regime $d \to \infty$, we prove under suitable assumptions that a phase transition occurs in the polynomial regime $n = d^κ$, with $n$ the sample size. Namely, we prove that $\lVert \hat Σ- E \hat Σ\rVert \Rightarrow 0$ if and only if $κ> κ_*$ for some threshold $κ_*$ determined by the behavior of the maximum likelihood ratios. Moreover, we identify general situations where $κ_* = 1/λ_1$, with $λ_1$ the smallest eigenvalue of the covariance matrix of the auxiliary distribution used to estimate $\hat Σ$ by importance sampling. This suggests that importance sampling will work better with covariance matrices having a large smallest eigenvalue. We carry this insight into recent CE schemes proposed to estimate the probability of high-dimensional rare events. Through numerical simulations, we demonstrate that better CE schemes are also the ones with larger smallest eigenvalue, even though these algorithms were not designed to smooth the spectrum. This new spectral interpretation raises stimulating questions and opens research directions for the design of efficient high-dimensional algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11351 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Phase transition for conditional covariance matrices estimated by importance sampling, and implications for cross-entropy schemes in high dimension Beh, Jason Morio, Jerome Simatos, Florian Statistics Theory Probability Motivated by the estimation of covariance matrices by importance sampling arising in the cross-entropy (CE) algorithm, we study a random matrix model $\hat Σ= {\bf X} L {\bf X}^\top$ with two distinct features: $\bf X$ and $L$ are dependent, and $L$ is heavy-tailed. In the high-dimensional regime $d \to \infty$, we prove under suitable assumptions that a phase transition occurs in the polynomial regime $n = d^κ$, with $n$ the sample size. Namely, we prove that $\lVert \hat Σ- E \hat Σ\rVert \Rightarrow 0$ if and only if $κ> κ_*$ for some threshold $κ_*$ determined by the behavior of the maximum likelihood ratios. Moreover, we identify general situations where $κ_* = 1/λ_1$, with $λ_1$ the smallest eigenvalue of the covariance matrix of the auxiliary distribution used to estimate $\hat Σ$ by importance sampling. This suggests that importance sampling will work better with covariance matrices having a large smallest eigenvalue. We carry this insight into recent CE schemes proposed to estimate the probability of high-dimensional rare events. Through numerical simulations, we demonstrate that better CE schemes are also the ones with larger smallest eigenvalue, even though these algorithms were not designed to smooth the spectrum. This new spectral interpretation raises stimulating questions and opens research directions for the design of efficient high-dimensional algorithms. |
| title | Phase transition for conditional covariance matrices estimated by importance sampling, and implications for cross-entropy schemes in high dimension |
| topic | Statistics Theory Probability |
| url | https://arxiv.org/abs/2511.11351 |