Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.12074 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913117096640512 |
|---|---|
| author | Knab, Patrick Xhelili, Orgest Buzi, Inis Nilo, Drago Andres Guggiana Khan, Mohd Saquib Kolb, Lorenz Scherzer, Manuel Yildirir, Kerem Bartelt, Christian Schubert, Philipp Johannes |
| author_facet | Knab, Patrick Xhelili, Orgest Buzi, Inis Nilo, Drago Andres Guggiana Khan, Mohd Saquib Kolb, Lorenz Scherzer, Manuel Yildirir, Kerem Bartelt, Christian Schubert, Philipp Johannes |
| contents | Scene understanding is central to general physical intelligence, and video is a primary modality for capturing both state and temporal dynamics of a scene. Yet understanding physical processes remains difficult, as models must combine object localization, hand-object interactions, relational parsing, temporal reasoning, and step-level procedural inference. Existing benchmarks usually evaluate these capabilities separately, limiting diagnosis of why models fail on procedural tasks. We introduce BARISTA, a densely annotated egocentric dataset and benchmark of 185 real-world coffee-preparation videos covering fully automatic, portafilter-based, and capsule-based workflows. BARISTA provides verified per-frame scene graphs linking persistent object identities to masks, tracks, boxes, attributes, typed relations, hand-object interactions, activities, and process steps. From these graphs, we derive zero-shot language-based tasks spanning phrase grounding, hand-object interaction recognition, referring, activity recognition, relation extraction, and temporal visual question answering. Experiments reveal strong variation across task families and no consistently dominant model family, positioning BARISTA as a challenging diagnostic benchmark for procedural video understanding. Code and dataset available at https://huggingface.co/datasets/ramblr/BARISTA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_12074 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding Knab, Patrick Xhelili, Orgest Buzi, Inis Nilo, Drago Andres Guggiana Khan, Mohd Saquib Kolb, Lorenz Scherzer, Manuel Yildirir, Kerem Bartelt, Christian Schubert, Philipp Johannes Computer Vision and Pattern Recognition Scene understanding is central to general physical intelligence, and video is a primary modality for capturing both state and temporal dynamics of a scene. Yet understanding physical processes remains difficult, as models must combine object localization, hand-object interactions, relational parsing, temporal reasoning, and step-level procedural inference. Existing benchmarks usually evaluate these capabilities separately, limiting diagnosis of why models fail on procedural tasks. We introduce BARISTA, a densely annotated egocentric dataset and benchmark of 185 real-world coffee-preparation videos covering fully automatic, portafilter-based, and capsule-based workflows. BARISTA provides verified per-frame scene graphs linking persistent object identities to masks, tracks, boxes, attributes, typed relations, hand-object interactions, activities, and process steps. From these graphs, we derive zero-shot language-based tasks spanning phrase grounding, hand-object interaction recognition, referring, activity recognition, relation extraction, and temporal visual question answering. Experiments reveal strong variation across task families and no consistently dominant model family, positioning BARISTA as a challenging diagnostic benchmark for procedural video understanding. Code and dataset available at https://huggingface.co/datasets/ramblr/BARISTA. |
| title | BARISTA: A Multi-Task Egocentric Benchmark for Compositional Visual Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2605.12074 |