Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910267024080896 |
|---|---|
| author | Raugel, Joséphine Seitzer, Maximilian Szafraniec, Marc Vo, Huy V. Rapin, Jérémy Labatut, Patrick Bojanowski, Piotr Wyart, Valentin King, Jean-Rémi |
| author_facet | Raugel, Joséphine Seitzer, Maximilian Szafraniec, Marc Vo, Huy V. Rapin, Jérémy Labatut, Patrick Bojanowski, Piotr Wyart, Valentin King, Jean-Rémi |
| contents | Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains highly debated. In particular, while forward activations of pretrained models reliably map onto the cortical hierarchy of visual processing, it is unknown whether backpropagated gradients exhibit a similar correspondence. Here, we address this question using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) recordings of human brain responses to natural images. For this, we extend standard encoding analyses of forward activations to map backpropagated gradients onto neural data. Focusing on a recent self-supervised vision model (DINOv3) and reproducing results on eight vision models, we find that backpropagated gradients can reliably predict both fMRI and MEG signals, specifically in higher-level visual cortex and for later latencies. However, the spatial and temporal organization of these backpropagated gradients in the brain diverges from the patterns expected under a biologically plausible backpropagation mechanism: specifically, both the order in which gradients are computed and their spatial organization diverge from the temporal and spatial hierarchies of the human brain. Together, these results suggest that, although deep networks and the brain may share similar representational content, they likely rely on fundamentally different mechanisms to learn those representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_28693 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images Raugel, Joséphine Seitzer, Maximilian Szafraniec, Marc Vo, Huy V. Rapin, Jérémy Labatut, Patrick Bojanowski, Piotr Wyart, Valentin King, Jean-Rémi Neurons and Cognition Artificial Intelligence Backpropagation is the core learning mechanism underlying deep learning. However, whether and how this algorithm is implemented in the brain remains highly debated. In particular, while forward activations of pretrained models reliably map onto the cortical hierarchy of visual processing, it is unknown whether backpropagated gradients exhibit a similar correspondence. Here, we address this question using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) recordings of human brain responses to natural images. For this, we extend standard encoding analyses of forward activations to map backpropagated gradients onto neural data. Focusing on a recent self-supervised vision model (DINOv3) and reproducing results on eight vision models, we find that backpropagated gradients can reliably predict both fMRI and MEG signals, specifically in higher-level visual cortex and for later latencies. However, the spatial and temporal organization of these backpropagated gradients in the brain diverges from the patterns expected under a biologically plausible backpropagation mechanism: specifically, both the order in which gradients are computed and their spatial organization diverge from the temporal and spatial hierarchies of the human brain. Together, these results suggest that, although deep networks and the brain may share similar representational content, they likely rely on fundamentally different mechanisms to learn those representations. |
| title | Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images |
| topic | Neurons and Cognition Artificial Intelligence |
| url | https://arxiv.org/abs/2605.28693 |