Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2412.17227 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915075928883200 |
|---|---|
| author | Willett, Francis R. Li, Jingyuan Le, Trung Fan, Chaofei Chen, Mingfei Shlizerman, Eli Chen, Yue Zheng, Xin Okubo, Tatsuo S. Benster, Tyler Lee, Hyun Dong Kounga, Maxwell Buchanan, E. Kelly Zoltowski, David Linderman, Scott W. Henderson, Jaimie M. |
| author_facet | Willett, Francis R. Li, Jingyuan Le, Trung Fan, Chaofei Chen, Mingfei Shlizerman, Eli Chen, Yue Zheng, Xin Okubo, Tatsuo S. Benster, Tyler Lee, Hyun Dong Kounga, Maxwell Buchanan, E. Kelly Zoltowski, David Linderman, Scott W. Henderson, Jaimie M. |
| contents | Speech brain-computer interfaces aim to decipher what a person is trying to say from neural activity alone, restoring communication to people with paralysis who have lost the ability to speak intelligibly. The Brain-to-Text Benchmark '24 and associated competition was created to foster the advancement of decoding algorithms that convert neural activity to text. Here, we summarize the lessons learned from the competition ending on June 1, 2024 (the top 4 entrants also presented their experiences in a recorded webinar). The largest improvements in accuracy were achieved using an ensembling approach, where the output of multiple independent decoders was merged using a fine-tuned large language model (an approach used by all 3 top entrants). Performance gains were also found by improving how the baseline recurrent neural network (RNN) model was trained, including by optimizing learning rate scheduling and by using a diphone training objective. Improving upon the model architecture itself proved more difficult, however, with attempts to use deep state space models or transformers not yet appearing to offer a benefit over the RNN baseline. The benchmark will remain open indefinitely to support further work towards increasing the accuracy of brain-to-text algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17227 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Brain-to-Text Benchmark '24: Lessons Learned Willett, Francis R. Li, Jingyuan Le, Trung Fan, Chaofei Chen, Mingfei Shlizerman, Eli Chen, Yue Zheng, Xin Okubo, Tatsuo S. Benster, Tyler Lee, Hyun Dong Kounga, Maxwell Buchanan, E. Kelly Zoltowski, David Linderman, Scott W. Henderson, Jaimie M. Computation and Language Machine Learning Neurons and Cognition Speech brain-computer interfaces aim to decipher what a person is trying to say from neural activity alone, restoring communication to people with paralysis who have lost the ability to speak intelligibly. The Brain-to-Text Benchmark '24 and associated competition was created to foster the advancement of decoding algorithms that convert neural activity to text. Here, we summarize the lessons learned from the competition ending on June 1, 2024 (the top 4 entrants also presented their experiences in a recorded webinar). The largest improvements in accuracy were achieved using an ensembling approach, where the output of multiple independent decoders was merged using a fine-tuned large language model (an approach used by all 3 top entrants). Performance gains were also found by improving how the baseline recurrent neural network (RNN) model was trained, including by optimizing learning rate scheduling and by using a diphone training objective. Improving upon the model architecture itself proved more difficult, however, with attempts to use deep state space models or transformers not yet appearing to offer a benefit over the RNN baseline. The benchmark will remain open indefinitely to support further work towards increasing the accuracy of brain-to-text algorithms. |
| title | Brain-to-Text Benchmark '24: Lessons Learned |
| topic | Computation and Language Machine Learning Neurons and Cognition |
| url | https://arxiv.org/abs/2412.17227 |