Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909885576249344 |
|---|---|
| author | Sung, Ching-Chih Suzuki, Shuntaro Chien, Francis Pingfan Sugiura, Komei Tsao, Yu |
| author_facet | Sung, Ching-Chih Suzuki, Shuntaro Chien, Francis Pingfan Sugiura, Komei Tsao, Yu |
| contents | Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing. Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics, primarily in the superior temporal and inferior frontal gyri. However, previous studies have been largely confined to clean speech, leaving it unclear whether the brain employs condition-invariant neural codes across diverse listening environments. To address this gap, we propose a novel architecture built upon a deep state space model for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure. We present the first attempt to decode intelligibility across acoustically distinct conditions, showing our method significantly outperforms classical approaches. Furthermore, region-wise analysis highlights contributions from auditory, frontal, and parietal regions, and cross-condition transfer indicates the presence of condition-invariant neural codes, thereby advancing understanding of abstract linguistic representations in the brain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01868 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model Sung, Ching-Chih Suzuki, Shuntaro Chien, Francis Pingfan Sugiura, Komei Tsao, Yu Neurons and Cognition Machine Learning Sound Audio and Speech Processing Signal Processing Clarifying the neural basis of speech intelligibility is critical for computational neuroscience and digital speech processing. Recent neuroimaging studies have shown that intelligibility modulates cortical activity beyond simple acoustics, primarily in the superior temporal and inferior frontal gyri. However, previous studies have been largely confined to clean speech, leaving it unclear whether the brain employs condition-invariant neural codes across diverse listening environments. To address this gap, we propose a novel architecture built upon a deep state space model for decoding intelligibility from fMRI signals, specifically tailored to their high-dimensional temporal structure. We present the first attempt to decode intelligibility across acoustically distinct conditions, showing our method significantly outperforms classical approaches. Furthermore, region-wise analysis highlights contributions from auditory, frontal, and parietal regions, and cross-condition transfer indicates the presence of condition-invariant neural codes, thereby advancing understanding of abstract linguistic representations in the brain. |
| title | Condition-Invariant fMRI Decoding of Speech Intelligibility with Deep State Space Model |
| topic | Neurons and Cognition Machine Learning Sound Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2511.01868 |