EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913409004470272 |
|---|---|
| author | Pei, Baoqi Chen, Guo Xu, Jilan He, Yuping Liu, Yicheng Pan, Kanghua Huang, Yifei Wang, Yali Lu, Tong Wang, Limin Qiao, Yu |
| author_facet | Pei, Baoqi Chen, Guo Xu, Jilan He, Yuping Liu, Yicheng Pan, Kanghua Huang, Yifei Wang, Yali Lu, Tong Wang, Limin Qiao, Yu |
| contents | In this report, we present our solutions to the EgoVis Challenges in CVPR 2024, including five tracks in the Ego4D challenge and three tracks in the EPIC-Kitchens challenge. Building upon the video-language two-tower model and leveraging our meticulously organized egocentric video data, we introduce a novel foundation model called EgoVideo. This model is specifically designed to cater to the unique characteristics of egocentric videos and provides strong support for our competition submissions. In the Ego4D challenges, we tackle various tasks including Natural Language Queries, Step Grounding, Moment Queries, Short-term Object Interaction Anticipation, and Long-term Action Anticipation. In addition, we also participate in the EPIC-Kitchens challenge, where we engage in the Action Recognition, Multiple Instance Retrieval, and Domain Adaptation for Action Recognition tracks. By adapting EgoVideo to these diverse tasks, we showcase its versatility and effectiveness in different egocentric video analysis scenarios, demonstrating the powerful representation ability of EgoVideo as an egocentric foundation model. Our codebase and pretrained models are publicly available at https://github.com/OpenGVLab/EgoVideo. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18070 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation Pei, Baoqi Chen, Guo Xu, Jilan He, Yuping Liu, Yicheng Pan, Kanghua Huang, Yifei Wang, Yali Lu, Tong Wang, Limin Qiao, Yu Computer Vision and Pattern Recognition In this report, we present our solutions to the EgoVis Challenges in CVPR 2024, including five tracks in the Ego4D challenge and three tracks in the EPIC-Kitchens challenge. Building upon the video-language two-tower model and leveraging our meticulously organized egocentric video data, we introduce a novel foundation model called EgoVideo. This model is specifically designed to cater to the unique characteristics of egocentric videos and provides strong support for our competition submissions. In the Ego4D challenges, we tackle various tasks including Natural Language Queries, Step Grounding, Moment Queries, Short-term Object Interaction Anticipation, and Long-term Action Anticipation. In addition, we also participate in the EPIC-Kitchens challenge, where we engage in the Action Recognition, Multiple Instance Retrieval, and Domain Adaptation for Action Recognition tracks. By adapting EgoVideo to these diverse tasks, we showcase its versatility and effectiveness in different egocentric video analysis scenarios, demonstrating the powerful representation ability of EgoVideo as an egocentric foundation model. Our codebase and pretrained models are publicly available at https://github.com/OpenGVLab/EgoVideo. |
| title | EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.18070 |