On the Error-Propagation of Inexact Hotelling's Deflation for Principal Component Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917678218739712 |
|---|---|
| author | Liao, Fangshuo Kim, Junhyung Lyle Barnum, Cruz Kyrillidis, Anastasios |
| author_facet | Liao, Fangshuo Kim, Junhyung Lyle Barnum, Cruz Kyrillidis, Anastasios |
| contents | Principal Component Analysis (PCA) aims to find subspaces spanned by the so-called principal components that best represent the variance in the dataset. The deflation method is a popular meta-algorithm that sequentially finds individual principal components, starting from the most important ones and working towards the less important ones. However, as deflation proceeds, numerical errors from the imprecise estimation of principal components propagate due to its sequential nature. This paper mathematically characterizes the error propagation of the inexact Hotelling's deflation method. We consider two scenarios: $i)$ when the sub-routine for finding the leading eigenvector is abstract and can represent various algorithms; and $ii)$ when power iteration is used as the sub-routine. In the latter case, the additional directional information from power iteration allows us to obtain a tighter error bound than the sub-routine agnostic case. For both scenarios, we explicitly characterize how the errors progress and affect subsequent principal component estimations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_04283 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | On the Error-Propagation of Inexact Hotelling's Deflation for Principal Component Analysis Liao, Fangshuo Kim, Junhyung Lyle Barnum, Cruz Kyrillidis, Anastasios Machine Learning Optimization and Control Principal Component Analysis (PCA) aims to find subspaces spanned by the so-called principal components that best represent the variance in the dataset. The deflation method is a popular meta-algorithm that sequentially finds individual principal components, starting from the most important ones and working towards the less important ones. However, as deflation proceeds, numerical errors from the imprecise estimation of principal components propagate due to its sequential nature. This paper mathematically characterizes the error propagation of the inexact Hotelling's deflation method. We consider two scenarios: $i)$ when the sub-routine for finding the leading eigenvector is abstract and can represent various algorithms; and $ii)$ when power iteration is used as the sub-routine. In the latter case, the additional directional information from power iteration allows us to obtain a tighter error bound than the sub-routine agnostic case. For both scenarios, we explicitly characterize how the errors progress and affect subsequent principal component estimations. |
| title | On the Error-Propagation of Inexact Hotelling's Deflation for Principal Component Analysis |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2310.04283 |