Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918037976776704 |
|---|---|
| author | Nie, Jingping Tran, Dung T. Thakkar, Karan Kowtha, Vasudha Huang, Jon Avendano, Carlos Azemi, Erdrin Mitra, Vikramjit |
| author_facet | Nie, Jingping Tran, Dung T. Thakkar, Karan Kowtha, Vasudha Huang, Jon Avendano, Carlos Azemi, Erdrin Mitra, Vikramjit |
| contents | Auscultation, particularly heart sound, is a non-invasive technique that provides essential vital sign information. Recently, self-supervised acoustic representation foundation models (FMs) have been proposed to offer insights into acoustics-based vital signs. However, there has been little exploration of the extent to which auscultation is encoded in these pre-trained FM representations. In this work, using a publicly available phonocardiogram (PCG) dataset and a heart rate (HR) estimation model, we conduct a layer-wise investigation of six acoustic representation FMs: HuBERT, wav2vec2, wavLM, Whisper, Contrastive Language-Audio Pretraining (CLAP), and an in-house CLAP model. Additionally, we implement the baseline method from Nie et al., 2024 (which relies on acoustic features) and show that overall, representation vectors from pre-trained foundation models (FMs) offer comparable performance to the baseline. Notably, HR estimation using the representations from the audio encoder of the in-house CLAP model outperforms the results obtained from the baseline, achieving a lower mean absolute error (MAE) across various train/validation/test splits despite the domain mismatch. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20745 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation Nie, Jingping Tran, Dung T. Thakkar, Karan Kowtha, Vasudha Huang, Jon Avendano, Carlos Azemi, Erdrin Mitra, Vikramjit Sound Machine Learning Audio and Speech Processing Auscultation, particularly heart sound, is a non-invasive technique that provides essential vital sign information. Recently, self-supervised acoustic representation foundation models (FMs) have been proposed to offer insights into acoustics-based vital signs. However, there has been little exploration of the extent to which auscultation is encoded in these pre-trained FM representations. In this work, using a publicly available phonocardiogram (PCG) dataset and a heart rate (HR) estimation model, we conduct a layer-wise investigation of six acoustic representation FMs: HuBERT, wav2vec2, wavLM, Whisper, Contrastive Language-Audio Pretraining (CLAP), and an in-house CLAP model. Additionally, we implement the baseline method from Nie et al., 2024 (which relies on acoustic features) and show that overall, representation vectors from pre-trained foundation models (FMs) offer comparable performance to the baseline. Notably, HR estimation using the representations from the audio encoder of the in-house CLAP model outperforms the results obtained from the baseline, achieving a lower mean absolute error (MAE) across various train/validation/test splits despite the domain mismatch. |
| title | Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.20745 |