Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.02846 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911298058452992 |
|---|---|
| author | Benavent-Lledo, Manuel Bacharidis, Konstantinos Manousaki, Victoria Papoutsakis, Konstantinos Argyros, Antonis Garcia-Rodriguez, Jose |
| author_facet | Benavent-Lledo, Manuel Bacharidis, Konstantinos Manousaki, Victoria Papoutsakis, Konstantinos Argyros, Antonis Garcia-Rodriguez, Jose |
| contents | Anticipating actions before they occur is a core challenge in action understanding research. While conventional methods rely on extracting and aggregating temporal information from videos, as humans we can often predict upcoming actions by observing a single moment from a scene, when given sufficient context. Can a model achieve this competence? The short answer is yes, although its effectiveness depends on the complexity of the task. In this work, we investigate to what extent video aggregation can be replaced with alternative modalities. To this end, based on recent advances in visual feature extraction and language-based reasoning, we introduce AAG, a method for Action Anticipation at a Glimpse. AAG combines RGB features with depth cues from a single frame for enhanced spatial reasoning, and incorporates prior action information to provide long-term context. This context is obtained either through textual summaries from Vision-Language Models, or from predictions generated by a single-frame action recognizer. Our results demonstrate that multimodal single-frame action anticipation using AAG can perform competitively compared to both temporally aggregated video baselines and state-of-the-art methods across three instructional activity datasets: IKEA-ASM, Meccano, and Assembly101. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_02846 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video? Benavent-Lledo, Manuel Bacharidis, Konstantinos Manousaki, Victoria Papoutsakis, Konstantinos Argyros, Antonis Garcia-Rodriguez, Jose Computer Vision and Pattern Recognition Anticipating actions before they occur is a core challenge in action understanding research. While conventional methods rely on extracting and aggregating temporal information from videos, as humans we can often predict upcoming actions by observing a single moment from a scene, when given sufficient context. Can a model achieve this competence? The short answer is yes, although its effectiveness depends on the complexity of the task. In this work, we investigate to what extent video aggregation can be replaced with alternative modalities. To this end, based on recent advances in visual feature extraction and language-based reasoning, we introduce AAG, a method for Action Anticipation at a Glimpse. AAG combines RGB features with depth cues from a single frame for enhanced spatial reasoning, and incorporates prior action information to provide long-term context. This context is obtained either through textual summaries from Vision-Language Models, or from predictions generated by a single-frame action recognizer. Our results demonstrate that multimodal single-frame action anticipation using AAG can perform competitively compared to both temporally aggregated video baselines and state-of-the-art methods across three instructional activity datasets: IKEA-ASM, Meccano, and Assembly101. |
| title | Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video? |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.02846 |