Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.04535 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908304376070144 |
|---|---|
| author | Lin, Weikai Ma, Tianrui Boloor, Adith Feng, Yu Xing, Ruofan Zhang, Xuan Zhu, Yuhao |
| author_facet | Lin, Weikai Ma, Tianrui Boloor, Adith Feng, Yu Xing, Ruofan Zhang, Xuan Zhu, Yuhao |
| contents | Energy-efficient image acquisition on the edge is crucial for enabling remote sensing applications where the sensor node has weak compute capabilities and must transmit data to a remote server/cloud for processing. To reduce the edge energy consumption, this paper proposes a sensor-algorithm co-designed system called SnapPix, which compresses raw pixels in the analog domain inside the sensor. We use coded exposure (CE) as the in-sensor compression strategy as it offers the flexibility to sample, i.e., selectively expose pixels, both spatially and temporally. SNAPPIX has three contributions. First, we propose a task-agnostic strategy to learn the sampling/exposure pattern based on the classic theory of efficient coding. Second, we co-design the downstream vision model with the exposure pattern to address the pixel-level non-uniformity unique to CE-compressed images. Finally, we propose lightweight augmentations to the image sensor hardware to support our in-sensor CE compression. Evaluating on action recognition and video reconstruction, SnapPix outperforms state-of-the-art video-based methods at the same speed while reducing the energy by up to 15.4x. We have open-sourced the code at: https://github.com/horizon-research/SnapPix. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_04535 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SnapPix: Efficient-Coding--Inspired In-Sensor Compression for Edge Vision Lin, Weikai Ma, Tianrui Boloor, Adith Feng, Yu Xing, Ruofan Zhang, Xuan Zhu, Yuhao Computer Vision and Pattern Recognition Artificial Intelligence I.2 Energy-efficient image acquisition on the edge is crucial for enabling remote sensing applications where the sensor node has weak compute capabilities and must transmit data to a remote server/cloud for processing. To reduce the edge energy consumption, this paper proposes a sensor-algorithm co-designed system called SnapPix, which compresses raw pixels in the analog domain inside the sensor. We use coded exposure (CE) as the in-sensor compression strategy as it offers the flexibility to sample, i.e., selectively expose pixels, both spatially and temporally. SNAPPIX has three contributions. First, we propose a task-agnostic strategy to learn the sampling/exposure pattern based on the classic theory of efficient coding. Second, we co-design the downstream vision model with the exposure pattern to address the pixel-level non-uniformity unique to CE-compressed images. Finally, we propose lightweight augmentations to the image sensor hardware to support our in-sensor CE compression. Evaluating on action recognition and video reconstruction, SnapPix outperforms state-of-the-art video-based methods at the same speed while reducing the energy by up to 15.4x. We have open-sourced the code at: https://github.com/horizon-research/SnapPix. |
| title | SnapPix: Efficient-Coding--Inspired In-Sensor Compression for Edge Vision |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence I.2 |
| url | https://arxiv.org/abs/2504.04535 |