Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2406.08090 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917774440267776 |
|---|---|
| author | Zhang, Ziran Ma, Yongrui Chen, Yueting Zhang, Feng Gu, Jinwei Xue, Tianfan Guo, Shi |
| author_facet | Zhang, Ziran Ma, Yongrui Chen, Yueting Zhang, Feng Gu, Jinwei Xue, Tianfan Guo, Shi |
| contents | Video Frame Interpolation (VFI) is important for video enhancement, frame rate up-conversion, and slow-motion generation. The introduction of event cameras, which capture per-pixel brightness changes asynchronously, has significantly enhanced VFI capabilities, particularly for high-speed, nonlinear motions. However, these event-based methods encounter challenges in low-light conditions, notably trailing artifacts and signal latency, which hinder their direct applicability and generalization. Addressing these issues, we propose a novel per-scene optimization strategy tailored for low-light conditions. This approach utilizes the internal statistics of a sequence to handle degraded event data under low-light conditions, improving the generalizability to different lighting and camera settings. To evaluate its robustness in low-light condition, we further introduce EVFI-LL, a unique RGB+Event dataset captured under low-light conditions. Our results demonstrate state-of-the-art performance in low-light environments. Project page: https://naturezhanghn.github.io/sim2real. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08090 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization Zhang, Ziran Ma, Yongrui Chen, Yueting Zhang, Feng Gu, Jinwei Xue, Tianfan Guo, Shi Computer Vision and Pattern Recognition Video Frame Interpolation (VFI) is important for video enhancement, frame rate up-conversion, and slow-motion generation. The introduction of event cameras, which capture per-pixel brightness changes asynchronously, has significantly enhanced VFI capabilities, particularly for high-speed, nonlinear motions. However, these event-based methods encounter challenges in low-light conditions, notably trailing artifacts and signal latency, which hinder their direct applicability and generalization. Addressing these issues, we propose a novel per-scene optimization strategy tailored for low-light conditions. This approach utilizes the internal statistics of a sequence to handle degraded event data under low-light conditions, improving the generalizability to different lighting and camera settings. To evaluate its robustness in low-light condition, we further introduce EVFI-LL, a unique RGB+Event dataset captured under low-light conditions. Our results demonstrate state-of-the-art performance in low-light environments. Project page: https://naturezhanghn.github.io/sim2real. |
| title | From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.08090 |