Hybrid Event Frame Sensors: Modeling, Calibration, and Simulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909917622829056 |
|---|---|
| author | Lu, Yunfan Messikommer, Nico Xu, Xiaogang Chen, Liming Chen, Yuhan Zubic, Nikola Scaramuzza, Davide Xiong, Hui |
| author_facet | Lu, Yunfan Messikommer, Nico Xu, Xiaogang Chen, Liming Chen, Yuhan Zubic, Nikola Scaramuzza, Davide Xiong, Hui |
| contents | Event frame hybrid sensors integrate an Active Pixel Sensor (APS) and an Event Vision Sensor (EVS) within a single chip, combining the high dynamic range and low latency of the EVS with the rich spatial intensity information from the APS. While this tight integration offers compact, temporally precise imaging, the complex circuit architecture introduces non-trivial noise patterns that remain poorly understood and unmodeled. In this work, we present the first unified, statistics-based imaging noise model that jointly describes the noise behavior of APS and EVS pixels. Our formulation explicitly incorporates photon shot noise, dark current noise, fixed-pattern noise, and quantization noise, and links EVS noise to illumination level and dark current. Based on this formulation, we further develop a calibration pipeline to estimate noise parameters from real data and offer a detailed analysis of both APS and EVS noise behaviors. Finally, we propose HESIM, a statistically grounded simulator that generates RAW frames and events under realistic, jointly calibrated noise statistics. Experiments on two hybrid sensors validate our model across multiple imaging tasks (e.g., video frame interpolation and deblurring), demonstrating strong transfer from simulation to real data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_18037 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Hybrid Event Frame Sensors: Modeling, Calibration, and Simulation Lu, Yunfan Messikommer, Nico Xu, Xiaogang Chen, Liming Chen, Yuhan Zubic, Nikola Scaramuzza, Davide Xiong, Hui Computer Vision and Pattern Recognition Event frame hybrid sensors integrate an Active Pixel Sensor (APS) and an Event Vision Sensor (EVS) within a single chip, combining the high dynamic range and low latency of the EVS with the rich spatial intensity information from the APS. While this tight integration offers compact, temporally precise imaging, the complex circuit architecture introduces non-trivial noise patterns that remain poorly understood and unmodeled. In this work, we present the first unified, statistics-based imaging noise model that jointly describes the noise behavior of APS and EVS pixels. Our formulation explicitly incorporates photon shot noise, dark current noise, fixed-pattern noise, and quantization noise, and links EVS noise to illumination level and dark current. Based on this formulation, we further develop a calibration pipeline to estimate noise parameters from real data and offer a detailed analysis of both APS and EVS noise behaviors. Finally, we propose HESIM, a statistically grounded simulator that generates RAW frames and events under realistic, jointly calibrated noise statistics. Experiments on two hybrid sensors validate our model across multiple imaging tasks (e.g., video frame interpolation and deblurring), demonstrating strong transfer from simulation to real data. |
| title | Hybrid Event Frame Sensors: Modeling, Calibration, and Simulation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.18037 |