Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2508.15473 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866912546776154112 |
|---|---|
| author | Sung, Ching-Chih Hsin, Cheng-Hung Shiah, Yu-Anne Lin, Bo-Jyun Lai, Yi-Xuan Lee, Chia-Ying Wang, Yu-Te Su, Borchin Tsao, Yu |
| author_facet | Sung, Ching-Chih Hsin, Cheng-Hung Shiah, Yu-Anne Lin, Bo-Jyun Lai, Yi-Xuan Lee, Chia-Ying Wang, Yu-Te Su, Borchin Tsao, Yu |
| contents | This paper presents EffortNet, a novel deep learning framework for decoding individual listening effort from electroencephalography (EEG) during speech comprehension. Listening effort represents a significant challenge in speech-hearing research, particularly for aging populations and those with hearing impairment. We collected 64-channel EEG data from 122 participants during speech comprehension under four conditions: clean, noisy, MMSE-enhanced, and Transformer-enhanced speech. Statistical analyses confirmed that alpha oscillations (8-13 Hz) exhibited significantly higher power during noisy speech processing compared to clean or enhanced conditions, confirming their validity as objective biomarkers of listening effort. To address the substantial inter-individual variability in EEG signals, EffortNet integrates three complementary learning paradigms: self-supervised learning to leverage unlabeled data, incremental learning for progressive adaptation to individual characteristics, and transfer learning for efficient knowledge transfer to new subjects. Our experimental results demonstrate that Effort- Net achieves 80.9% classification accuracy with only 40% training data from new subjects, significantly outperforming conventional CNN (62.3%) and STAnet (61.1%) models. The probability-based metric derived from our model revealed that Transformer-enhanced speech elicited neural responses more similar to clean speech than MMSEenhanced speech. This finding contrasted with subjective intelligibility ratings but aligned with objective metrics. The proposed framework provides a practical solution for personalized assessment of hearing technologies, with implications for designing cognitive-aware speech enhancement systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_15473 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EffortNet: A Deep Learning Framework for Objective Assessment of Speech Enhancement Technologies Using EEG-Based Alpha Oscillations Sung, Ching-Chih Hsin, Cheng-Hung Shiah, Yu-Anne Lin, Bo-Jyun Lai, Yi-Xuan Lee, Chia-Ying Wang, Yu-Te Su, Borchin Tsao, Yu Audio and Speech Processing This paper presents EffortNet, a novel deep learning framework for decoding individual listening effort from electroencephalography (EEG) during speech comprehension. Listening effort represents a significant challenge in speech-hearing research, particularly for aging populations and those with hearing impairment. We collected 64-channel EEG data from 122 participants during speech comprehension under four conditions: clean, noisy, MMSE-enhanced, and Transformer-enhanced speech. Statistical analyses confirmed that alpha oscillations (8-13 Hz) exhibited significantly higher power during noisy speech processing compared to clean or enhanced conditions, confirming their validity as objective biomarkers of listening effort. To address the substantial inter-individual variability in EEG signals, EffortNet integrates three complementary learning paradigms: self-supervised learning to leverage unlabeled data, incremental learning for progressive adaptation to individual characteristics, and transfer learning for efficient knowledge transfer to new subjects. Our experimental results demonstrate that Effort- Net achieves 80.9% classification accuracy with only 40% training data from new subjects, significantly outperforming conventional CNN (62.3%) and STAnet (61.1%) models. The probability-based metric derived from our model revealed that Transformer-enhanced speech elicited neural responses more similar to clean speech than MMSEenhanced speech. This finding contrasted with subjective intelligibility ratings but aligned with objective metrics. The proposed framework provides a practical solution for personalized assessment of hearing technologies, with implications for designing cognitive-aware speech enhancement systems. |
| title | EffortNet: A Deep Learning Framework for Objective Assessment of Speech Enhancement Technologies Using EEG-Based Alpha Oscillations |
| topic | Audio and Speech Processing |
| url | https://arxiv.org/abs/2508.15473 |