Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909625192808448 |
|---|---|
| author | Jalouzot, Louis Thual, Alexis Lakretz, Yair Pallier, Christophe Thirion, Bertrand |
| author_facet | Jalouzot, Louis Thual, Alexis Lakretz, Yair Pallier, Christophe Thirion, Bertrand |
| contents | We investigate optimal strategies for decoding perceived natural speech from fMRI data acquired from a limited number of participants. Leveraging Lebel et al. (2023)'s dataset of 8 participants, we first demonstrate the effectiveness of training deep neural networks to predict LLM-derived text representations from fMRI activity. Then, in this data regime, we observe that multi-subject training does not improve decoding accuracy compared to single-subject approach. Furthermore, training on similar or different stimuli across subjects has a negligible effect on decoding accuracy. Finally, we find that our decoders better model syntactic than semantic features, and that stories containing sentences with complex syntax or rich semantic content are more challenging to decode. While our results demonstrate the benefits of having extensive data per participant (deep phenotyping), they suggest that leveraging multi-subject for natural speech decoding likely requires deeper phenotyping or a substantially larger cohort. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21304 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants Jalouzot, Louis Thual, Alexis Lakretz, Yair Pallier, Christophe Thirion, Bertrand Neurons and Cognition Computation and Language Machine Learning We investigate optimal strategies for decoding perceived natural speech from fMRI data acquired from a limited number of participants. Leveraging Lebel et al. (2023)'s dataset of 8 participants, we first demonstrate the effectiveness of training deep neural networks to predict LLM-derived text representations from fMRI activity. Then, in this data regime, we observe that multi-subject training does not improve decoding accuracy compared to single-subject approach. Furthermore, training on similar or different stimuli across subjects has a negligible effect on decoding accuracy. Finally, we find that our decoders better model syntactic than semantic features, and that stories containing sentences with complex syntax or rich semantic content are more challenging to decode. While our results demonstrate the benefits of having extensive data per participant (deep phenotyping), they suggest that leveraging multi-subject for natural speech decoding likely requires deeper phenotyping or a substantially larger cohort. |
| title | Optimizing fMRI Data Acquisition for Decoding Natural Speech with Limited Participants |
| topic | Neurons and Cognition Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2505.21304 |