Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2305.11953 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908852585234432 |
|---|---|
| author | Wang, Siwei de Laittre, Elizabeth A MacLean, Jason Palmer, Stephanie E |
| author_facet | Wang, Siwei de Laittre, Elizabeth A MacLean, Jason Palmer, Stephanie E |
| contents | Representational drift refers to an unstable mapping between neural activity and input sensory or output behavioral variables. While much work has focused on the effect of representational drift on single, simple external variables, we investigate the differences in representational drift across spatiotemporal features in a moving visual stimulus. The neural responses across animals to the same movie reflect both common, encoded stimulus features and idiosyncratic individual variation. To extract the shared neural encoding of stimulus features only, we learn a latent space embedding using weakly supervised contrastive learning. This approach pulls neural activity together in the embedding space if they are responses to the same stimulus segment and push them apart if not. This approach enables us to probe how stimulus features fluctuating as fast as 33 ms (the movie frame rate) are encoded by variable neural codes across animals. It also allows us to investigate how representational drift changes the encoding in individuals across sessions. We observe that our learned embedding is near-optimal for decoding natural features (background scenery, local motion, complex spatio-temporal features, and time) and neural activity from novel animals. This suggests that our embedding retains the encoding of multiple features at higher temporal granularity compared to previous methods. To quantify representational drift, we apply the trained decoder (which achieves near-optimal performance in one session) to a subsequent session recorded 90 minutes later. We then use the decrease in decoding performance as a proxy for the magnitude of drift. We show that the drift changes the encoding of fast-varying local motion features at a rate 5-6 times higher than slower-varying scenery features. Drift also perturbs the local geometry in the embedding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_11953 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Representational drift changes the encoding of fast and slow-varying natural scene features differently Wang, Siwei de Laittre, Elizabeth A MacLean, Jason Palmer, Stephanie E Neurons and Cognition Representational drift refers to an unstable mapping between neural activity and input sensory or output behavioral variables. While much work has focused on the effect of representational drift on single, simple external variables, we investigate the differences in representational drift across spatiotemporal features in a moving visual stimulus. The neural responses across animals to the same movie reflect both common, encoded stimulus features and idiosyncratic individual variation. To extract the shared neural encoding of stimulus features only, we learn a latent space embedding using weakly supervised contrastive learning. This approach pulls neural activity together in the embedding space if they are responses to the same stimulus segment and push them apart if not. This approach enables us to probe how stimulus features fluctuating as fast as 33 ms (the movie frame rate) are encoded by variable neural codes across animals. It also allows us to investigate how representational drift changes the encoding in individuals across sessions. We observe that our learned embedding is near-optimal for decoding natural features (background scenery, local motion, complex spatio-temporal features, and time) and neural activity from novel animals. This suggests that our embedding retains the encoding of multiple features at higher temporal granularity compared to previous methods. To quantify representational drift, we apply the trained decoder (which achieves near-optimal performance in one session) to a subsequent session recorded 90 minutes later. We then use the decrease in decoding performance as a proxy for the magnitude of drift. We show that the drift changes the encoding of fast-varying local motion features at a rate 5-6 times higher than slower-varying scenery features. Drift also perturbs the local geometry in the embedding. |
| title | Representational drift changes the encoding of fast and slow-varying natural scene features differently |
| topic | Neurons and Cognition |
| url | https://arxiv.org/abs/2305.11953 |