AoE: Always-on Egocentric Human Video Collection for Embodied AI
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918365632659456 |
|---|---|
| author | Yang, Bowen Li, Zishuo Sun, Yang Miao, Changtao Yang, Yifan Luo, Man Yan, Xiaotong Jiang, Feng Shi, Jinchuan Fu, Yankai Chen, Ning Zhao, Junkai Wang, Pengwei Yao, Guocai Zhang, Shanghang Chen, Hao Li, Zhe Zhu, Kai |
| author_facet | Yang, Bowen Li, Zishuo Sun, Yang Miao, Changtao Yang, Yifan Luo, Man Yan, Xiaotong Jiang, Feng Shi, Jinchuan Fu, Yankai Chen, Ning Zhao, Junkai Wang, Pengwei Yao, Guocai Zhang, Shanghang Chen, Hao Li, Zhe Zhu, Kai |
| contents | Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high infrastructure costs, complex hardware dependencies, and limited interaction scope, making scalable expansion challenging. In fact, humans themselves are ideal physically embodied agents. Therefore, obtaining egocentric real-world interaction data from globally distributed "human agents" offers advantages of low cost and sustainability. To this end, we propose the Always-on Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones, enabling low-cost, highly efficient, and scene-agnostic real-world interaction data collection to address the challenge of data scarcity. Specifically, we first employ an ergonomic neck-mounted smartphone holder to enable low-barrier, large-scale egocentric data collection through a cloud-edge collaborative architecture. Second, we develop a cross-platform mobile APP that leverages on-device compute for real-time processing, while the cloud hosts automated labeling and filtering pipelines that transform raw videos into high-quality training data. Finally, the AoE system supports distributed Ego video data collection by anyone, anytime, and anywhere. We evaluate AoE on data preprocessing quality and downstream tasks, demonstrating that high-quality egocentric data significantly boosts real-world generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_23893 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AoE: Always-on Egocentric Human Video Collection for Embodied AI Yang, Bowen Li, Zishuo Sun, Yang Miao, Changtao Yang, Yifan Luo, Man Yan, Xiaotong Jiang, Feng Shi, Jinchuan Fu, Yankai Chen, Ning Zhao, Junkai Wang, Pengwei Yao, Guocai Zhang, Shanghang Chen, Hao Li, Zhe Zhu, Kai Computer Vision and Pattern Recognition Robotics Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high infrastructure costs, complex hardware dependencies, and limited interaction scope, making scalable expansion challenging. In fact, humans themselves are ideal physically embodied agents. Therefore, obtaining egocentric real-world interaction data from globally distributed "human agents" offers advantages of low cost and sustainability. To this end, we propose the Always-on Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones, enabling low-cost, highly efficient, and scene-agnostic real-world interaction data collection to address the challenge of data scarcity. Specifically, we first employ an ergonomic neck-mounted smartphone holder to enable low-barrier, large-scale egocentric data collection through a cloud-edge collaborative architecture. Second, we develop a cross-platform mobile APP that leverages on-device compute for real-time processing, while the cloud hosts automated labeling and filtering pipelines that transform raw videos into high-quality training data. Finally, the AoE system supports distributed Ego video data collection by anyone, anytime, and anywhere. We evaluate AoE on data preprocessing quality and downstream tasks, demonstrating that high-quality egocentric data significantly boosts real-world generalization. |
| title | AoE: Always-on Egocentric Human Video Collection for Embodied AI |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2602.23893 |