Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2308.11771 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914645607972864 |
|---|---|
| author | Chen, Qinyu Wang, Zuowen Liu, Shih-Chii Gao, Chang |
| author_facet | Chen, Qinyu Wang, Zuowen Liu, Shih-Chii Gao, Chang |
| contents | This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7$\times$ without losing accuracy when tested on a \texttt{v2e}-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at \url{https://github.com/qinche106/cb-convlstm-eyetracking}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_11771 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | 3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network Chen, Qinyu Wang, Zuowen Liu, Shih-Chii Gao, Chang Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning 68T07, 68T10 This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7$\times$ without losing accuracy when tested on a \texttt{v2e}-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at \url{https://github.com/qinche106/cb-convlstm-eyetracking}. |
| title | 3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning 68T07, 68T10 |
| url | https://arxiv.org/abs/2308.11771 |