AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910150403555328 |
|---|---|
| author | Yang, Huzheng Gee, James Shi, Jianbo |
| author_facet | Yang, Huzheng Gee, James Shi, Jianbo |
| contents | We study the intriguing connection between visual data, deep networks, and the brain. Our method creates a universal channel alignment by using brain voxel fMRI response prediction as the training objective. We discover that deep networks, trained with different objectives, share common feature channels across various models. These channels can be clustered into recurring sets, corresponding to distinct brain regions, indicating the formation of visual concepts. Tracing the clusters of channel responses onto the images, we see semantically meaningful object segments emerge, even without any supervised decoder. Furthermore, the universal feature alignment and the clustering of channels produce a picture and quantification of how visual information is processed through the different network layers, which produces precise comparisons between the networks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18344 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space Yang, Huzheng Gee, James Shi, Jianbo Computer Vision and Pattern Recognition We study the intriguing connection between visual data, deep networks, and the brain. Our method creates a universal channel alignment by using brain voxel fMRI response prediction as the training objective. We discover that deep networks, trained with different objectives, share common feature channels across various models. These channels can be clustered into recurring sets, corresponding to distinct brain regions, indicating the formation of visual concepts. Tracing the clusters of channel responses onto the images, we see semantically meaningful object segments emerge, even without any supervised decoder. Furthermore, the universal feature alignment and the clustering of channels produce a picture and quantification of how visual information is processed through the different network layers, which produces precise comparisons between the networks. |
| title | AlignedCut: Visual Concepts Discovery on Brain-Guided Universal Feature Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.18344 |