Brain-to-Text Decoding: A Non-invasive Approach via Typing
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916628772421632 |
|---|---|
| author | Lévy, Jarod Zhang, Mingfang Pinet, Svetlana Rapin, Jérémy Banville, Hubert d'Ascoli, Stéphane King, Jean-Rémi |
| author_facet | Lévy, Jarod Zhang, Mingfang Pinet, Svetlana Rapin, Jérémy Banville, Hubert d'Ascoli, Stéphane King, Jean-Rémi |
| contents | Modern neuroprostheses can now restore communication in patients who have lost the ability to speak or move. However, these invasive devices entail risks inherent to neurosurgery. Here, we introduce a non-invasive method to decode the production of sentences from brain activity and demonstrate its efficacy in a cohort of 35 healthy volunteers. For this, we present Brain2Qwerty, a new deep learning architecture trained to decode sentences from either electro- (EEG) or magneto-encephalography (MEG), while participants typed briefly memorized sentences on a QWERTY keyboard. With MEG, Brain2Qwerty reaches, on average, a character-error-rate (CER) of 32% and substantially outperforms EEG (CER: 67%). For the best participants, the model achieves a CER of 19%, and can perfectly decode a variety of sentences outside of the training set. While error analyses suggest that decoding depends on motor processes, the analysis of typographical errors suggests that it also involves higher-level cognitive factors. Overall, these results narrow the gap between invasive and non-invasive methods and thus open the path for developing safe brain-computer interfaces for non-communicating patients. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17480 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Brain-to-Text Decoding: A Non-invasive Approach via Typing Lévy, Jarod Zhang, Mingfang Pinet, Svetlana Rapin, Jérémy Banville, Hubert d'Ascoli, Stéphane King, Jean-Rémi Signal Processing Artificial Intelligence Computation and Language Human-Computer Interaction Modern neuroprostheses can now restore communication in patients who have lost the ability to speak or move. However, these invasive devices entail risks inherent to neurosurgery. Here, we introduce a non-invasive method to decode the production of sentences from brain activity and demonstrate its efficacy in a cohort of 35 healthy volunteers. For this, we present Brain2Qwerty, a new deep learning architecture trained to decode sentences from either electro- (EEG) or magneto-encephalography (MEG), while participants typed briefly memorized sentences on a QWERTY keyboard. With MEG, Brain2Qwerty reaches, on average, a character-error-rate (CER) of 32% and substantially outperforms EEG (CER: 67%). For the best participants, the model achieves a CER of 19%, and can perfectly decode a variety of sentences outside of the training set. While error analyses suggest that decoding depends on motor processes, the analysis of typographical errors suggests that it also involves higher-level cognitive factors. Overall, these results narrow the gap between invasive and non-invasive methods and thus open the path for developing safe brain-computer interfaces for non-communicating patients. |
| title | Brain-to-Text Decoding: A Non-invasive Approach via Typing |
| topic | Signal Processing Artificial Intelligence Computation and Language Human-Computer Interaction |
| url | https://arxiv.org/abs/2502.17480 |