Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912715609473024 |
|---|---|
| author | Oono, Kenta Charoenphakdee, Nontawat Bito, Kotatsu Gao, Zhengyan Igata, Hideyoshi Yoshikawa, Masashi Ota, Yoshiaki Okui, Hiroki Akita, Kei Yamaguchi, Shoichiro Sugawara, Yohei Maeda, Shin-ichi Miyoshi, Kunihiko Saito, Yuki Tsuda, Koki Maruyama, Hiroshi Hayashi, Kohei |
| author_facet | Oono, Kenta Charoenphakdee, Nontawat Bito, Kotatsu Gao, Zhengyan Igata, Hideyoshi Yoshikawa, Masashi Ota, Yoshiaki Okui, Hiroki Akita, Kei Yamaguchi, Shoichiro Sugawara, Yohei Maeda, Shin-ichi Miyoshi, Kunihiko Saito, Yuki Tsuda, Koki Maruyama, Hiroshi Hayashi, Kohei |
| contents | Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the core algorithm, VHGM-MAE, a masked autoencoder (MAE) tailored for handling high-dimensional, sparse healthcare data. VHGM-MAE tackles four key technical challenges: (1) heterogeneity of healthcare data types, (2) probability distribution modeling, (3) systematic missingness in the training dataset arising from multiple data sources, and (4) the high-dimensional, small-$n$-large-$p$ problem. To address these challenges, VHGM-MAE employs a likelihood-based approach to model distributions with heterogeneous types, a transformer-based MAE to capture complex dependencies among observed and missing attributes, and a novel training scheme that effectively leverages available samples with diverse missingness patterns to mitigate the small-n-large-p problem. Experimental results demonstrate that VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_10656 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics Oono, Kenta Charoenphakdee, Nontawat Bito, Kotatsu Gao, Zhengyan Igata, Hideyoshi Yoshikawa, Masashi Ota, Yoshiaki Okui, Hiroki Akita, Kei Yamaguchi, Shoichiro Sugawara, Yohei Maeda, Shin-ichi Miyoshi, Kunihiko Saito, Yuki Tsuda, Koki Maruyama, Hiroshi Hayashi, Kohei Machine Learning Artificial Intelligence Virtual Human Generative Model (VHGM) is a generative model that approximates the joint probability over more than 2000 human healthcare-related attributes. This paper presents the core algorithm, VHGM-MAE, a masked autoencoder (MAE) tailored for handling high-dimensional, sparse healthcare data. VHGM-MAE tackles four key technical challenges: (1) heterogeneity of healthcare data types, (2) probability distribution modeling, (3) systematic missingness in the training dataset arising from multiple data sources, and (4) the high-dimensional, small-$n$-large-$p$ problem. To address these challenges, VHGM-MAE employs a likelihood-based approach to model distributions with heterogeneous types, a transformer-based MAE to capture complex dependencies among observed and missing attributes, and a novel training scheme that effectively leverages available samples with diverse missingness patterns to mitigate the small-n-large-p problem. Experimental results demonstrate that VHGM-MAE outperforms existing methods in both missing value imputation and synthetic data generation. |
| title | Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2306.10656 |