Neural Encoding and Decoding at Scale
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918033247698944 |
|---|---|
| author | Zhang, Yizi Wang, Yanchen Azabou, Mehdi Andre, Alexandre Wang, Zixuan Lyu, Hanrui Laboratory, The International Brain Dyer, Eva Paninski, Liam Hurwitz, Cole |
| author_facet | Zhang, Yizi Wang, Yanchen Azabou, Mehdi Andre, Alexandre Wang, Zixuan Lyu, Hanrui Laboratory, The International Brain Dyer, Eva Paninski, Liam Hurwitz, Cole |
| contents | Recent work has demonstrated that large-scale, multi-animal models are powerful tools for characterizing the relationship between neural activity and behavior. Current large-scale approaches, however, focus exclusively on either predicting neural activity from behavior (encoding) or predicting behavior from neural activity (decoding), limiting their ability to capture the bidirectional relationship between neural activity and behavior. To bridge this gap, we introduce a multimodal, multi-task model that enables simultaneous Neural Encoding and Decoding at Scale (NEDS). Central to our approach is a novel multi-task-masking strategy, which alternates between neural, behavioral, within-modality, and cross-modality masking. We pretrain our method on the International Brain Laboratory (IBL) repeated site dataset, which includes recordings from 83 animals performing the same visual decision-making task. In comparison to other large-scale models, we demonstrate that NEDS achieves state-of-the-art performance for both encoding and decoding when pretrained on multi-animal data and then fine-tuned on new animals. Surprisingly, NEDS's learned embeddings exhibit emergent properties: even without explicit training, they are highly predictive of the brain regions in each recording. Altogether, our approach is a step towards a foundation model of the brain that enables seamless translation between neural activity and behavior. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08201 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Neural Encoding and Decoding at Scale Zhang, Yizi Wang, Yanchen Azabou, Mehdi Andre, Alexandre Wang, Zixuan Lyu, Hanrui Laboratory, The International Brain Dyer, Eva Paninski, Liam Hurwitz, Cole Neurons and Cognition Artificial Intelligence Machine Learning Recent work has demonstrated that large-scale, multi-animal models are powerful tools for characterizing the relationship between neural activity and behavior. Current large-scale approaches, however, focus exclusively on either predicting neural activity from behavior (encoding) or predicting behavior from neural activity (decoding), limiting their ability to capture the bidirectional relationship between neural activity and behavior. To bridge this gap, we introduce a multimodal, multi-task model that enables simultaneous Neural Encoding and Decoding at Scale (NEDS). Central to our approach is a novel multi-task-masking strategy, which alternates between neural, behavioral, within-modality, and cross-modality masking. We pretrain our method on the International Brain Laboratory (IBL) repeated site dataset, which includes recordings from 83 animals performing the same visual decision-making task. In comparison to other large-scale models, we demonstrate that NEDS achieves state-of-the-art performance for both encoding and decoding when pretrained on multi-animal data and then fine-tuned on new animals. Surprisingly, NEDS's learned embeddings exhibit emergent properties: even without explicit training, they are highly predictive of the brain regions in each recording. Altogether, our approach is a step towards a foundation model of the brain that enables seamless translation between neural activity and behavior. |
| title | Neural Encoding and Decoding at Scale |
| topic | Neurons and Cognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2504.08201 |