Tracking objects that change in appearance with phase synchrony
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912254268538880 |
|---|---|
| author | Muzellec, Sabine Linsley, Drew Ashok, Alekh K. Mingolla, Ennio Malik, Girik VanRullen, Rufin Serre, Thomas |
| author_facet | Muzellec, Sabine Linsley, Drew Ashok, Alekh K. Mingolla, Ennio Malik, Girik VanRullen, Rufin Serre, Thomas |
| contents | Objects we encounter often change appearance as we interact with them. Changes in illumination (shadows), object pose, or the movement of non-rigid objects can drastically alter available image features. How do biological visual systems track objects as they change? One plausible mechanism involves attentional mechanisms for reasoning about the locations of objects independently of their appearances -- a capability that prominent neuroscience theories have associated with computing through neural synchrony. Here, we describe a novel deep learning circuit that can learn to precisely control attention to features separately from their location in the world through neural synchrony: the complex-valued recurrent neural network (CV-RNN). Next, we compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs), using FeatureTracker: a large-scale challenge that asks observers to track objects as their locations and appearances change in precisely controlled ways. While humans effortlessly solved FeatureTracker, state-of-the-art DNNs did not. In contrast, our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization as a neural substrate for tracking appearance-morphing objects as they move about. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_02094 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Tracking objects that change in appearance with phase synchrony Muzellec, Sabine Linsley, Drew Ashok, Alekh K. Mingolla, Ennio Malik, Girik VanRullen, Rufin Serre, Thomas Artificial Intelligence Computer Vision and Pattern Recognition Neurons and Cognition Objects we encounter often change appearance as we interact with them. Changes in illumination (shadows), object pose, or the movement of non-rigid objects can drastically alter available image features. How do biological visual systems track objects as they change? One plausible mechanism involves attentional mechanisms for reasoning about the locations of objects independently of their appearances -- a capability that prominent neuroscience theories have associated with computing through neural synchrony. Here, we describe a novel deep learning circuit that can learn to precisely control attention to features separately from their location in the world through neural synchrony: the complex-valued recurrent neural network (CV-RNN). Next, we compare object tracking in humans, the CV-RNN, and other deep neural networks (DNNs), using FeatureTracker: a large-scale challenge that asks observers to track objects as their locations and appearances change in precisely controlled ways. While humans effortlessly solved FeatureTracker, state-of-the-art DNNs did not. In contrast, our CV-RNN behaved similarly to humans on the challenge, providing a computational proof-of-concept for the role of phase synchronization as a neural substrate for tracking appearance-morphing objects as they move about. |
| title | Tracking objects that change in appearance with phase synchrony |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition Neurons and Cognition |
| url | https://arxiv.org/abs/2410.02094 |