Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.21080 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913913772179456 |
|---|---|
| author | Chowdhury, Sanjoy Biswas, Subrata Nag, Sayan Nagarajan, Tushar Murdock, Calvin Ananthabhotla, Ishwarya Qian, Yijun Ithapu, Vamsi Krishna Manocha, Dinesh Gao, Ruohan |
| author_facet | Chowdhury, Sanjoy Biswas, Subrata Nag, Sayan Nagarajan, Tushar Murdock, Calvin Ananthabhotla, Ishwarya Qian, Yijun Ithapu, Vamsi Krishna Manocha, Dinesh Gao, Ruohan |
| contents | Modern perception models, particularly those designed for multisensory egocentric tasks, have achieved remarkable performance but often come with substantial computational costs. These high demands pose challenges for real-world deployment, especially in resource-constrained environments. In this paper, we introduce EgoAdapt, a framework that adaptively performs cross-modal distillation and policy learning to enable efficient inference across different egocentric perception tasks, including egocentric action recognition, active speaker localization, and behavior anticipation. Our proposed policy module is adaptable to task-specific action spaces, making it broadly applicable. Experimental results on three challenging egocentric datasets EPIC-Kitchens, EasyCom, and Aria Everyday Activities demonstrate that our method significantly enhances efficiency, reducing GMACs by up to 89.09%, parameters up to 82.02%, and energy up to 9.6x, while still on-par and in many cases outperforming, the performance of corresponding state-of-the-art models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_21080 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception Chowdhury, Sanjoy Biswas, Subrata Nag, Sayan Nagarajan, Tushar Murdock, Calvin Ananthabhotla, Ishwarya Qian, Yijun Ithapu, Vamsi Krishna Manocha, Dinesh Gao, Ruohan Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Modern perception models, particularly those designed for multisensory egocentric tasks, have achieved remarkable performance but often come with substantial computational costs. These high demands pose challenges for real-world deployment, especially in resource-constrained environments. In this paper, we introduce EgoAdapt, a framework that adaptively performs cross-modal distillation and policy learning to enable efficient inference across different egocentric perception tasks, including egocentric action recognition, active speaker localization, and behavior anticipation. Our proposed policy module is adaptable to task-specific action spaces, making it broadly applicable. Experimental results on three challenging egocentric datasets EPIC-Kitchens, EasyCom, and Aria Everyday Activities demonstrate that our method significantly enhances efficiency, reducing GMACs by up to 89.09%, parameters up to 82.02%, and energy up to 9.6x, while still on-par and in many cases outperforming, the performance of corresponding state-of-the-art models. |
| title | EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perception |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2506.21080 |