Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.06042 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917562981285888 |
|---|---|
| author | Do, Tu T. Vu, Mai Anh Vo, Tuan L. Ly, Hoang Thien Nguyen, Thu Hicks, Steven A. Riegler, Michael A. Halvorsen, Pål Nguyen, Binh T. |
| author_facet | Do, Tu T. Vu, Mai Anh Vo, Tuan L. Ly, Hoang Thien Nguyen, Thu Hicks, Steven A. Riegler, Michael A. Halvorsen, Pål Nguyen, Binh T. |
| contents | Monotone missing data is a common problem in data analysis. However, imputation combined with dimensionality reduction can be computationally expensive, especially with the increasing size of datasets. To address this issue, we propose a Blockwise principal component analysis Imputation (BPI) framework for dimensionality reduction and imputation of monotone missing data. The framework conducts Principal Component Analysis (PCA) on the observed part of each monotone block of the data and then imputes on merging the obtained principal components using a chosen imputation technique. BPI can work with various imputation techniques and can significantly reduce imputation time compared to conducting dimensionality reduction after imputation. This makes it a practical and efficient approach for large datasets with monotone missing data. Our experiments validate the improvement in speed. In addition, our experiments also show that while applying MICE imputation directly on missing data may not yield convergence, applying BPI with MICE for the data may lead to convergence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_06042 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Blockwise Principal Component Analysis for monotone missing data imputation and dimensionality reduction Do, Tu T. Vu, Mai Anh Vo, Tuan L. Ly, Hoang Thien Nguyen, Thu Hicks, Steven A. Riegler, Michael A. Halvorsen, Pål Nguyen, Binh T. Machine Learning Monotone missing data is a common problem in data analysis. However, imputation combined with dimensionality reduction can be computationally expensive, especially with the increasing size of datasets. To address this issue, we propose a Blockwise principal component analysis Imputation (BPI) framework for dimensionality reduction and imputation of monotone missing data. The framework conducts Principal Component Analysis (PCA) on the observed part of each monotone block of the data and then imputes on merging the obtained principal components using a chosen imputation technique. BPI can work with various imputation techniques and can significantly reduce imputation time compared to conducting dimensionality reduction after imputation. This makes it a practical and efficient approach for large datasets with monotone missing data. Our experiments validate the improvement in speed. In addition, our experiments also show that while applying MICE imputation directly on missing data may not yield convergence, applying BPI with MICE for the data may lead to convergence. |
| title | Blockwise Principal Component Analysis for monotone missing data imputation and dimensionality reduction |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.06042 |