Asymptotic Normality of Generalized Low-Rank Matrix Sensing via Riemannian Geometry
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909491633586176 |
|---|---|
| author | Bastani, Osbert |
| author_facet | Bastani, Osbert |
| contents | We prove an asymptotic normality guarantee for generalized low-rank matrix sensing -- i.e., matrix sensing under a general convex loss $\bar\ell(\langle X,M\rangle,y^*)$, where $M\in\mathbb{R}^{d\times d}$ is the unknown rank-$k$ matrix, $X$ is a measurement matrix, and $y^*$ is the corresponding measurement. Our analysis relies on tools from Riemannian geometry to handle degeneracy of the Hessian of the loss due to rotational symmetry in the parameter space. In particular, we parameterize the manifold of low-rank matrices by $\barθ\barθ^\top$, where $\barθ\in\mathbb{R}^{d\times k}$. Then, assuming the minimizer of the empirical loss $\barθ^0\in\mathbb{R}^{d\times k}$ is in a constant size ball around the true parameters $\barθ^*$, we prove $\sqrt{n}(ϕ^0-ϕ^*)\xrightarrow{D}N(0,(H^*)^{-1})$ as $n\to\infty$, where $ϕ^0$ and $ϕ^*$ are representations of $\barθ^*$ and $\barθ^0$ in the horizontal space of the Riemannian quotient manifold $\mathbb{R}^{d\times k}/\text{O}(k)$, and $H^*$ is the Hessian of the true loss in the same representation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_10238 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Asymptotic Normality of Generalized Low-Rank Matrix Sensing via Riemannian Geometry Bastani, Osbert Machine Learning We prove an asymptotic normality guarantee for generalized low-rank matrix sensing -- i.e., matrix sensing under a general convex loss $\bar\ell(\langle X,M\rangle,y^*)$, where $M\in\mathbb{R}^{d\times d}$ is the unknown rank-$k$ matrix, $X$ is a measurement matrix, and $y^*$ is the corresponding measurement. Our analysis relies on tools from Riemannian geometry to handle degeneracy of the Hessian of the loss due to rotational symmetry in the parameter space. In particular, we parameterize the manifold of low-rank matrices by $\barθ\barθ^\top$, where $\barθ\in\mathbb{R}^{d\times k}$. Then, assuming the minimizer of the empirical loss $\barθ^0\in\mathbb{R}^{d\times k}$ is in a constant size ball around the true parameters $\barθ^*$, we prove $\sqrt{n}(ϕ^0-ϕ^*)\xrightarrow{D}N(0,(H^*)^{-1})$ as $n\to\infty$, where $ϕ^0$ and $ϕ^*$ are representations of $\barθ^*$ and $\barθ^0$ in the horizontal space of the Riemannian quotient manifold $\mathbb{R}^{d\times k}/\text{O}(k)$, and $H^*$ is the Hessian of the true loss in the same representation. |
| title | Asymptotic Normality of Generalized Low-Rank Matrix Sensing via Riemannian Geometry |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.10238 |