Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2401.01461 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911746509242368 |
|---|---|
| author | Wu, Xiaotong Lai, Wei-Sheng Shih, YiChang Herrmann, Charles Krainin, Michael Sun, Deqing Liang, Chia-Kai |
| author_facet | Wu, Xiaotong Lai, Wei-Sheng Shih, YiChang Herrmann, Charles Krainin, Michael Sun, Deqing Liang, Chia-Kai |
| contents | DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems crop and digitally upsample images from W, leading to significant detail loss. In this paper, we propose an efficient system for hybrid zoom super-resolution on mobile devices, which captures a synchronous pair of W and T shots and leverages machine learning models to align and transfer details from T to W. We further develop an adaptive blending method that accounts for depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment errors. To minimize the domain gap, we design a dual-phone camera rig to capture real-world inputs and ground-truths for supervised training. Our method generates a 12-megapixel image in 500ms on a mobile platform and compares favorably against state-of-the-art methods under extensive evaluation on real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_01461 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Efficient Hybrid Zoom using Camera Fusion on Mobile Phones Wu, Xiaotong Lai, Wei-Sheng Shih, YiChang Herrmann, Charles Krainin, Michael Sun, Deqing Liang, Chia-Kai Computer Vision and Pattern Recognition DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types. However, these techniques are not possible on smartphone devices due to space constraints. Most smartphone manufacturers adopt a hybrid zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T) camera at a high zoom level. To simulate zoom levels between W and T, these systems crop and digitally upsample images from W, leading to significant detail loss. In this paper, we propose an efficient system for hybrid zoom super-resolution on mobile devices, which captures a synchronous pair of W and T shots and leverages machine learning models to align and transfer details from T to W. We further develop an adaptive blending method that accounts for depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment errors. To minimize the domain gap, we design a dual-phone camera rig to capture real-world inputs and ground-truths for supervised training. Our method generates a 12-megapixel image in 500ms on a mobile platform and compares favorably against state-of-the-art methods under extensive evaluation on real-world scenarios. |
| title | Efficient Hybrid Zoom using Camera Fusion on Mobile Phones |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.01461 |