Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.04965 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917780860698624 |
|---|---|
| author | Xu, Xiran Wang, Bo Yan, Yujie Zhu, Haolin Zhang, Zechen Wu, Xihong Chen, Jing |
| author_facet | Xu, Xiran Wang, Bo Yan, Yujie Zhu, Haolin Zhang, Zechen Wu, Xihong Chen, Jing |
| contents | To investigate the processing of speech in the brain, simple linear models are commonly used to establish a relationship between brain signals and speech features. However, these linear models are ill-equipped to model a highly dynamic and complex non-linear system like the brain. Although non-linear methods with neural networks have been developed recently, reconstructing unseen stimuli from unseen subjects' EEG is still a highly challenging task. This work presents a novel method, ConvConcatNet, to reconstruct mel-specgrams from EEG, in which the deep convolution neural network and extensive concatenation operation were combined. With our ConvConcatNet model, the Pearson correlation between the reconstructed and the target mel-spectrogram can achieve 0.0420, which was ranked as No.1 in the Task 2 of the Auditory EEG Challenge. The codes and models to implement our work will be available on Github: https://github.com/xuxiran/ConvConcatNet |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_04965 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG Xu, Xiran Wang, Bo Yan, Yujie Zhu, Haolin Zhang, Zechen Wu, Xihong Chen, Jing Signal Processing Machine Learning To investigate the processing of speech in the brain, simple linear models are commonly used to establish a relationship between brain signals and speech features. However, these linear models are ill-equipped to model a highly dynamic and complex non-linear system like the brain. Although non-linear methods with neural networks have been developed recently, reconstructing unseen stimuli from unseen subjects' EEG is still a highly challenging task. This work presents a novel method, ConvConcatNet, to reconstruct mel-specgrams from EEG, in which the deep convolution neural network and extensive concatenation operation were combined. With our ConvConcatNet model, the Pearson correlation between the reconstructed and the target mel-spectrogram can achieve 0.0420, which was ranked as No.1 in the Task 2 of the Auditory EEG Challenge. The codes and models to implement our work will be available on Github: https://github.com/xuxiran/ConvConcatNet |
| title | ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG |
| topic | Signal Processing Machine Learning |
| url | https://arxiv.org/abs/2401.04965 |