Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914567912685568 |
|---|---|
| author | Nagrani, Arsha Uijilings, Jasper Buch, Shyamal Weyand, Tobias Vijayanarasimhan, Sudheendra Hu, Bo Mehran, Ramin Ross, David A Schmid, Cordelia |
| author_facet | Nagrani, Arsha Uijilings, Jasper Buch, Shyamal Weyand, Tobias Vijayanarasimhan, Sudheendra Hu, Bo Mehran, Ramin Ross, David A Schmid, Cordelia |
| contents | Video reasoning models are a core component of egocentric and embodied agents. However, standard benchmarks for assessing models provide only evaluation of the output (e.g. the answer to a question), without evaluation of intermediate reasoning steps, and most provide answers only in the text domain. We introduce Minerva-Ego, a benchmark for evaluating complex egocentric visual reasoning. We extend recent high-quality video data sources recorded from egocentric / embodied settings with a set of challenging, multi-step multimodal questions and spatiotemporally-dense human-annotated reasoning traces. Benchmarking experiments show that state-of-the-art models still have a large gap to human performance. To investigate this gap in detail, we annotate each reasoning trace in the dataset with the objects of interest required to solve the question, as spatiotemporal mask annotations. Through extensive evaluations, we identify that prompting frontier models with hints of 'where' and 'when' to look yields substantial improvements in performance. Minerva-Ego can be downloaded at https://github.com/google-deepmind/neptune. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_15342 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding Nagrani, Arsha Uijilings, Jasper Buch, Shyamal Weyand, Tobias Vijayanarasimhan, Sudheendra Hu, Bo Mehran, Ramin Ross, David A Schmid, Cordelia Computer Vision and Pattern Recognition Machine Learning Video reasoning models are a core component of egocentric and embodied agents. However, standard benchmarks for assessing models provide only evaluation of the output (e.g. the answer to a question), without evaluation of intermediate reasoning steps, and most provide answers only in the text domain. We introduce Minerva-Ego, a benchmark for evaluating complex egocentric visual reasoning. We extend recent high-quality video data sources recorded from egocentric / embodied settings with a set of challenging, multi-step multimodal questions and spatiotemporally-dense human-annotated reasoning traces. Benchmarking experiments show that state-of-the-art models still have a large gap to human performance. To investigate this gap in detail, we annotate each reasoning trace in the dataset with the objects of interest required to solve the question, as spatiotemporal mask annotations. Through extensive evaluations, we identify that prompting frontier models with hints of 'where' and 'when' to look yields substantial improvements in performance. Minerva-Ego can be downloaded at https://github.com/google-deepmind/neptune. |
| title | Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding |
| topic | Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2605.15342 |