Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.15473 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Table of Contents:
- This paper presents EffortNet, a novel deep learning framework for decoding individual listening effort from electroencephalography (EEG) during speech comprehension. Listening effort represents a significant challenge in speech-hearing research, particularly for aging populations and those with hearing impairment. We collected 64-channel EEG data from 122 participants during speech comprehension under four conditions: clean, noisy, MMSE-enhanced, and Transformer-enhanced speech. Statistical analyses confirmed that alpha oscillations (8-13 Hz) exhibited significantly higher power during noisy speech processing compared to clean or enhanced conditions, confirming their validity as objective biomarkers of listening effort. To address the substantial inter-individual variability in EEG signals, EffortNet integrates three complementary learning paradigms: self-supervised learning to leverage unlabeled data, incremental learning for progressive adaptation to individual characteristics, and transfer learning for efficient knowledge transfer to new subjects. Our experimental results demonstrate that Effort- Net achieves 80.9% classification accuracy with only 40% training data from new subjects, significantly outperforming conventional CNN (62.3%) and STAnet (61.1%) models. The probability-based metric derived from our model revealed that Transformer-enhanced speech elicited neural responses more similar to clean speech than MMSEenhanced speech. This finding contrasted with subjective intelligibility ratings but aligned with objective metrics. The proposed framework provides a practical solution for personalized assessment of hearing technologies, with implications for designing cognitive-aware speech enhancement systems.