Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.06561 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914432359071744 |
|---|---|
| author | Zhu, Fangrui Xi, Yunfeng Ni, Jianmo Cai, Mu Gong, Boqing Zhao, Long Qu, Chen Miao, Ian Li, Yi Zhong, Cheng Jiang, Huaizu Patel, Shwetak |
| author_facet | Zhu, Fangrui Xi, Yunfeng Ni, Jianmo Cai, Mu Gong, Boqing Zhao, Long Qu, Chen Miao, Ian Li, Yi Zhong, Cheng Jiang, Huaizu Patel, Shwetak |
| contents | Egocentric video understanding is inherently complex due to the dynamic 4D nature of the environment, where camera motion and object displacements necessitate a continuous re-evaluation of spatial relations. In this work, we target a suite of under-explored egocentric 4D reasoning tasks, including fixture interaction counting, viewpoint-relative fixture location, object movement itinerary tracking, and stationary object localization, that require fundamentally different cognitive operations: spatial anchoring, temporal tracking, and duration reasoning. We observe that these structural differences make task-agnostic approaches insufficient: generic Chain-of-Thought methods lack task-appropriate reasoning primitives, and uniform reinforcement learning actively destabilizes performance on spatial tasks. To address this, we propose EgoReasoner, a two-stage framework that aligns both the reasoning scaffold and the reward signal to each task's cognitive structure. In the first stage, Task-Adaptive Thinking Templates guide the synthesis of structured CoT traces that teach the model to reason adaptively across task types via supervised fine-tuning. In the second stage, task-aware reward functions verify entity grounding, temporal alignment, and task-adaptive logical consistency, selectively strengthening each reasoning pathway via reinforcement fine-tuning with GRPO. Our 3B-parameter model, trained on only 16K samples, achieves 37.5% average accuracy on the challenging HD-EPIC benchmark, surpassing Qwen2.5-VL-7B (25.7%) by over 10 points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_06561 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking Zhu, Fangrui Xi, Yunfeng Ni, Jianmo Cai, Mu Gong, Boqing Zhao, Long Qu, Chen Miao, Ian Li, Yi Zhong, Cheng Jiang, Huaizu Patel, Shwetak Computer Vision and Pattern Recognition Egocentric video understanding is inherently complex due to the dynamic 4D nature of the environment, where camera motion and object displacements necessitate a continuous re-evaluation of spatial relations. In this work, we target a suite of under-explored egocentric 4D reasoning tasks, including fixture interaction counting, viewpoint-relative fixture location, object movement itinerary tracking, and stationary object localization, that require fundamentally different cognitive operations: spatial anchoring, temporal tracking, and duration reasoning. We observe that these structural differences make task-agnostic approaches insufficient: generic Chain-of-Thought methods lack task-appropriate reasoning primitives, and uniform reinforcement learning actively destabilizes performance on spatial tasks. To address this, we propose EgoReasoner, a two-stage framework that aligns both the reasoning scaffold and the reward signal to each task's cognitive structure. In the first stage, Task-Adaptive Thinking Templates guide the synthesis of structured CoT traces that teach the model to reason adaptively across task types via supervised fine-tuning. In the second stage, task-aware reward functions verify entity grounding, temporal alignment, and task-adaptive logical consistency, selectively strengthening each reasoning pathway via reinforcement fine-tuning with GRPO. Our 3B-parameter model, trained on only 16K samples, achieves 37.5% average accuracy on the challenging HD-EPIC benchmark, surpassing Qwen2.5-VL-7B (25.7%) by over 10 points. |
| title | EgoReasoner: Learning Egocentric 4D Reasoning via Task-Adaptive Structured Thinking |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.06561 |