EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908448299417600 |
|---|---|
| author | Peng, Zhengyuan Xu, Jianqing Li, Shen Ji, Jiazhen Huang, Yuge Zhang, Jingyun Li, Jinmin Ding, Shouhong Guo, Rizen Tan, Xin Ma, Lizhuang |
| author_facet | Peng, Zhengyuan Xu, Jianqing Li, Shen Ji, Jiazhen Huang, Yuge Zhang, Jingyun Li, Jinmin Ding, Shouhong Guo, Rizen Tan, Xin Ma, Lizhuang |
| contents | Human-machine interaction through augmented reality (AR) and virtual reality (VR) is increasingly prevalent, requiring accurate and efficient gaze estimation which hinges on the accuracy of eye segmentation to enable smooth user experiences. We introduce EyeSeg, a novel eye segmentation framework designed to overcome key challenges that existing approaches struggle with: motion blur, eyelid occlusion, and train-test domain gaps. In these situations, existing models struggle to extract robust features, leading to suboptimal performance. Noting that these challenges can be generally quantified by uncertainty, we design EyeSeg as an uncertainty-aware eye segmentation framework for AR/VR wherein we explicitly model the uncertainties by performing Bayesian uncertainty learning of a posterior under the closed set prior. Theoretically, we prove that a statistic of the learned posterior indicates segmentation uncertainty levels and empirically outperforms existing methods in downstream tasks, such as gaze estimation. EyeSeg outputs an uncertainty score and the segmentation result, weighting and fusing multiple gaze estimates for robustness, which proves to be effective especially under motion blur, eyelid occlusion and cross-domain challenges. Moreover, empirical results suggest that EyeSeg achieves segmentation improvements of MIoU, E1, F1, and ACC surpassing previous approaches. The code is publicly available at https://github.com/JethroPeng/EyeSeg. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09649 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR Peng, Zhengyuan Xu, Jianqing Li, Shen Ji, Jiazhen Huang, Yuge Zhang, Jingyun Li, Jinmin Ding, Shouhong Guo, Rizen Tan, Xin Ma, Lizhuang Computer Vision and Pattern Recognition Human-machine interaction through augmented reality (AR) and virtual reality (VR) is increasingly prevalent, requiring accurate and efficient gaze estimation which hinges on the accuracy of eye segmentation to enable smooth user experiences. We introduce EyeSeg, a novel eye segmentation framework designed to overcome key challenges that existing approaches struggle with: motion blur, eyelid occlusion, and train-test domain gaps. In these situations, existing models struggle to extract robust features, leading to suboptimal performance. Noting that these challenges can be generally quantified by uncertainty, we design EyeSeg as an uncertainty-aware eye segmentation framework for AR/VR wherein we explicitly model the uncertainties by performing Bayesian uncertainty learning of a posterior under the closed set prior. Theoretically, we prove that a statistic of the learned posterior indicates segmentation uncertainty levels and empirically outperforms existing methods in downstream tasks, such as gaze estimation. EyeSeg outputs an uncertainty score and the segmentation result, weighting and fusing multiple gaze estimates for robustness, which proves to be effective especially under motion blur, eyelid occlusion and cross-domain challenges. Moreover, empirical results suggest that EyeSeg achieves segmentation improvements of MIoU, E1, F1, and ACC surpassing previous approaches. The code is publicly available at https://github.com/JethroPeng/EyeSeg. |
| title | EyeSeg: An Uncertainty-Aware Eye Segmentation Framework for AR/VR |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.09649 |