From Minutes to Days: Scaling Intracranial Speech Decoding with Supervised Pretraining
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915684834869248 |
|---|---|
| author | Evanson, Linnea Zhang, Mingfang Banville, Hubert Panchavati, Saarang Bourdillon, Pierre King, Jean-Rémi |
| author_facet | Evanson, Linnea Zhang, Mingfang Banville, Hubert Panchavati, Saarang Bourdillon, Pierre King, Jean-Rémi |
| contents | Decoding speech from brain activity has typically relied on limited neural recordings collected during short and highly controlled experiments. Here, we introduce a framework to leverage week-long intracranial and audio recordings from patients undergoing clinical monitoring, effectively increasing the training dataset size by over two orders of magnitude. With this pretraining, our contrastive learning model substantially outperforms models trained solely on classic experimental data, with gains that scale log-linearly with dataset size. Analysis of the learned representations reveals that, while brain activity represents speech features, its global structure largely drifts across days, highlighting the need for models that explicitly account for cross-day variability. Overall, our approach opens a scalable path toward decoding and modeling brain representations in both real-life and controlled task settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_15830 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Minutes to Days: Scaling Intracranial Speech Decoding with Supervised Pretraining Evanson, Linnea Zhang, Mingfang Banville, Hubert Panchavati, Saarang Bourdillon, Pierre King, Jean-Rémi Sound Computation and Language Neurons and Cognition Decoding speech from brain activity has typically relied on limited neural recordings collected during short and highly controlled experiments. Here, we introduce a framework to leverage week-long intracranial and audio recordings from patients undergoing clinical monitoring, effectively increasing the training dataset size by over two orders of magnitude. With this pretraining, our contrastive learning model substantially outperforms models trained solely on classic experimental data, with gains that scale log-linearly with dataset size. Analysis of the learned representations reveals that, while brain activity represents speech features, its global structure largely drifts across days, highlighting the need for models that explicitly account for cross-day variability. Overall, our approach opens a scalable path toward decoding and modeling brain representations in both real-life and controlled task settings. |
| title | From Minutes to Days: Scaling Intracranial Speech Decoding with Supervised Pretraining |
| topic | Sound Computation and Language Neurons and Cognition |
| url | https://arxiv.org/abs/2512.15830 |