How Many Bytes Can You Take Out Of Brain-To-Text Decoding?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916257461174272 |
|---|---|
| author | Antonello, Richard Sarma, Nihita Tang, Jerry Song, Jiaru Huth, Alexander |
| author_facet | Antonello, Richard Sarma, Nihita Tang, Jerry Song, Jiaru Huth, Alexander |
| contents | Brain-computer interfaces have promising medical and scientific applications for aiding speech and studying the brain. In this work, we propose an information-based evaluation metric for brain-to-text decoders. Using this metric, we examine two methods to augment existing state-of-the-art continuous text decoders. We show that these methods, in concert, can improve brain decoding performance by upwards of 40% when compared to a baseline model. We further examine the informatic properties of brain-to-text decoders and show empirically that they have Zipfian power law dynamics. Finally, we provide an estimate for the idealized performance of an fMRI-based text decoder. We compare this idealized model to our current model, and use our information-based metric to quantify the main sources of decoding error. We conclude that a practical brain-to-text decoder is likely possible given further algorithmic improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_14055 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | How Many Bytes Can You Take Out Of Brain-To-Text Decoding? Antonello, Richard Sarma, Nihita Tang, Jerry Song, Jiaru Huth, Alexander Computation and Language Artificial Intelligence Emerging Technologies Brain-computer interfaces have promising medical and scientific applications for aiding speech and studying the brain. In this work, we propose an information-based evaluation metric for brain-to-text decoders. Using this metric, we examine two methods to augment existing state-of-the-art continuous text decoders. We show that these methods, in concert, can improve brain decoding performance by upwards of 40% when compared to a baseline model. We further examine the informatic properties of brain-to-text decoders and show empirically that they have Zipfian power law dynamics. Finally, we provide an estimate for the idealized performance of an fMRI-based text decoder. We compare this idealized model to our current model, and use our information-based metric to quantify the main sources of decoding error. We conclude that a practical brain-to-text decoder is likely possible given further algorithmic improvements. |
| title | How Many Bytes Can You Take Out Of Brain-To-Text Decoding? |
| topic | Computation and Language Artificial Intelligence Emerging Technologies |
| url | https://arxiv.org/abs/2405.14055 |