TeleEgo: Benchmarking Egocentric AI Assistants in the Wild
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909952591790080 |
|---|---|
| author | Yan, Jiaqi Ren, Ruilong Liu, Jingren Xu, Shuning Wang, Ling Wang, Yiheng Zhong, Xinlin Wang, Yun Zhang, Long Chen, Xiangyu Sun, Changzhi Luo, Jixiang Zhang, Dell Sun, Hao Zhang, Chi Li, Xuelong |
| author_facet | Yan, Jiaqi Ren, Ruilong Liu, Jingren Xu, Shuning Wang, Ling Wang, Yiheng Zhong, Xinlin Wang, Yun Zhang, Long Chen, Xiangyu Sun, Changzhi Luo, Jixiang Zhang, Dell Sun, Hao Zhang, Chi Li, Xuelong |
| contents | Egocentric AI assistants in real-world settings must process multi-modal inputs (video, audio, text), respond in real time, and retain evolving long-term memory. However, existing benchmarks typically evaluate these abilities in isolation, lack realistic streaming scenarios, or support only short-term tasks. We introduce \textbf{TeleEgo}, a long-duration, streaming, omni-modal benchmark for evaluating egocentric AI assistants in realistic daily contexts. The dataset features over 14 hours per participant of synchronized egocentric video, audio, and text across four domains: work \& study, lifestyle \& routines, social activities, and outings \& culture. All data is aligned on a unified global timeline and includes high-quality visual narrations and speech transcripts, curated through human refinement.TeleEgo defines 12 diagnostic subtasks across three core capabilities: Memory (recalling past events), Understanding (interpreting the current moment), and Cross-Memory Reasoning (linking distant events). It contains 3,291 human-verified QA items spanning multiple question formats (single-choice, binary, multi-choice, and open-ended), evaluated strictly in a streaming setting. We propose Real-Time Accuracy (RTA) to jointly capture correctness and responsiveness under tight decision windows, and Memory Persistence Time (MPT) as a forward-looking metric for long-term retention in continuous streams. In this work, we report RTA results for current models and release TeleEgo, together with an MPT evaluation framework, as a realistic and extensible benchmark for future egocentric assistants with stronger streaming memory, enabling systematic study of both real-time behavior and long-horizon memory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_23981 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TeleEgo: Benchmarking Egocentric AI Assistants in the Wild Yan, Jiaqi Ren, Ruilong Liu, Jingren Xu, Shuning Wang, Ling Wang, Yiheng Zhong, Xinlin Wang, Yun Zhang, Long Chen, Xiangyu Sun, Changzhi Luo, Jixiang Zhang, Dell Sun, Hao Zhang, Chi Li, Xuelong Computer Vision and Pattern Recognition Egocentric AI assistants in real-world settings must process multi-modal inputs (video, audio, text), respond in real time, and retain evolving long-term memory. However, existing benchmarks typically evaluate these abilities in isolation, lack realistic streaming scenarios, or support only short-term tasks. We introduce \textbf{TeleEgo}, a long-duration, streaming, omni-modal benchmark for evaluating egocentric AI assistants in realistic daily contexts. The dataset features over 14 hours per participant of synchronized egocentric video, audio, and text across four domains: work \& study, lifestyle \& routines, social activities, and outings \& culture. All data is aligned on a unified global timeline and includes high-quality visual narrations and speech transcripts, curated through human refinement.TeleEgo defines 12 diagnostic subtasks across three core capabilities: Memory (recalling past events), Understanding (interpreting the current moment), and Cross-Memory Reasoning (linking distant events). It contains 3,291 human-verified QA items spanning multiple question formats (single-choice, binary, multi-choice, and open-ended), evaluated strictly in a streaming setting. We propose Real-Time Accuracy (RTA) to jointly capture correctness and responsiveness under tight decision windows, and Memory Persistence Time (MPT) as a forward-looking metric for long-term retention in continuous streams. In this work, we report RTA results for current models and release TeleEgo, together with an MPT evaluation framework, as a realistic and extensible benchmark for future egocentric assistants with stronger streaming memory, enabling systematic study of both real-time behavior and long-horizon memory. |
| title | TeleEgo: Benchmarking Egocentric AI Assistants in the Wild |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.23981 |