CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910485705654272 |
|---|---|
| author | Shah, Sachin Chan, Matthew Albert Cai, Haoming Chen, Jingxi Kulshrestha, Sakshum Singh, Chahat Deep Aloimonos, Yiannis Metzler, Christopher |
| author_facet | Shah, Sachin Chan, Matthew Albert Cai, Haoming Chen, Jingxi Kulshrestha, Sakshum Singh, Chahat Deep Aloimonos, Yiannis Metzler, Christopher |
| contents | Point-spread-function (PSF) engineering is a well-established computational imaging technique that uses phase masks and other optical elements to embed extra information (e.g., depth) into the images captured by conventional CMOS image sensors. To date, however, PSF-engineering has not been applied to neuromorphic event cameras; a powerful new image sensing technology that responds to changes in the log-intensity of light.
This paper establishes theoretical limits (Cramér Rao bounds) on 3D point localization and tracking with PSF-engineered event cameras. Using these bounds, we first demonstrate that existing Fisher phase masks are already near-optimal for localizing static flashing point sources (e.g., blinking fluorescent molecules). We then demonstrate that existing designs are sub-optimal for tracking moving point sources and proceed to use our theory to design optimal phase masks and binary amplitude masks for this task. To overcome the non-convexity of the design problem, we leverage novel implicit neural representation based parameterizations of the phase and amplitude masks. We demonstrate the efficacy of our designs through extensive simulations. We also validate our method with a simple prototype. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_09409 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras Shah, Sachin Chan, Matthew Albert Cai, Haoming Chen, Jingxi Kulshrestha, Sakshum Singh, Chahat Deep Aloimonos, Yiannis Metzler, Christopher Computer Vision and Pattern Recognition Image and Video Processing Point-spread-function (PSF) engineering is a well-established computational imaging technique that uses phase masks and other optical elements to embed extra information (e.g., depth) into the images captured by conventional CMOS image sensors. To date, however, PSF-engineering has not been applied to neuromorphic event cameras; a powerful new image sensing technology that responds to changes in the log-intensity of light. This paper establishes theoretical limits (Cramér Rao bounds) on 3D point localization and tracking with PSF-engineered event cameras. Using these bounds, we first demonstrate that existing Fisher phase masks are already near-optimal for localizing static flashing point sources (e.g., blinking fluorescent molecules). We then demonstrate that existing designs are sub-optimal for tracking moving point sources and proceed to use our theory to design optimal phase masks and binary amplitude masks for this task. To overcome the non-convexity of the design problem, we leverage novel implicit neural representation based parameterizations of the phase and amplitude masks. We demonstrate the efficacy of our designs through extensive simulations. We also validate our method with a simple prototype. |
| title | CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras |
| topic | Computer Vision and Pattern Recognition Image and Video Processing |
| url | https://arxiv.org/abs/2406.09409 |