Net2Brain: A Toolbox to compare artificial vision models with human brain responses
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909771966185472 |
|---|---|
| author | Bersch, Domenic Dwivedi, Kshitij Vilas, Martina Cichy, Radoslaw M. Roig, Gemma |
| author_facet | Bersch, Domenic Dwivedi, Kshitij Vilas, Martina Cichy, Radoslaw M. Roig, Gemma |
| contents | We introduce Net2Brain, a graphical and command-line user interface toolbox for comparing the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. While different toolboxes facilitate only single functionalities or only focus on a small subset of supervised image classification models, Net2Brain allows the extraction of activations of more than 600 DNNs trained to perform a diverse range of vision-related tasks (e.g semantic segmentation, depth estimation, action recognition, etc.), over both image and video datasets. The toolbox computes the representational dissimilarity matrices (RDMs) over those activations and compares them to brain recordings using representational similarity analysis (RSA), weighted RSA, both in specific ROIs and with searchlight search. In addition, it is possible to add a new data set of stimuli and brain recordings to the toolbox for evaluation. We demonstrate the functionality and advantages of Net2Brain with an example showcasing how it can be used to test hypotheses of cognitive computational neuroscience. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_09677 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Net2Brain: A Toolbox to compare artificial vision models with human brain responses Bersch, Domenic Dwivedi, Kshitij Vilas, Martina Cichy, Radoslaw M. Roig, Gemma Computer Vision and Pattern Recognition Artificial Intelligence Neurons and Cognition We introduce Net2Brain, a graphical and command-line user interface toolbox for comparing the representational spaces of artificial deep neural networks (DNNs) and human brain recordings. While different toolboxes facilitate only single functionalities or only focus on a small subset of supervised image classification models, Net2Brain allows the extraction of activations of more than 600 DNNs trained to perform a diverse range of vision-related tasks (e.g semantic segmentation, depth estimation, action recognition, etc.), over both image and video datasets. The toolbox computes the representational dissimilarity matrices (RDMs) over those activations and compares them to brain recordings using representational similarity analysis (RSA), weighted RSA, both in specific ROIs and with searchlight search. In addition, it is possible to add a new data set of stimuli and brain recordings to the toolbox for evaluation. We demonstrate the functionality and advantages of Net2Brain with an example showcasing how it can be used to test hypotheses of cognitive computational neuroscience. |
| title | Net2Brain: A Toolbox to compare artificial vision models with human brain responses |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Neurons and Cognition |
| url | https://arxiv.org/abs/2208.09677 |