SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908434409979904 |
|---|---|
| author | Hasson, Yana Luc, Pauline Momeni, Liliane Ovsjanikov, Maks Moing, Guillaume Le Kuznetsova, Alina Ktena, Ira Sun, Jennifer J. Koppula, Skanda Gokay, Dilara Heyward, Joseph Pot, Etienne Zisserman, Andrew |
| author_facet | Hasson, Yana Luc, Pauline Momeni, Liliane Ovsjanikov, Maks Moing, Guillaume Le Kuznetsova, Alina Ktena, Ira Sun, Jennifer J. Koppula, Skanda Gokay, Dilara Heyward, Joseph Pot, Etienne Zisserman, Andrew |
| contents | In recent years, there has been a proliferation of spatiotemporal foundation models in different scientific disciplines. While promising, these models are often domain-specific and are only assessed within the particular applications for which they are designed. Given that many tasks can be represented as video modeling problems, video foundation models (ViFMs) hold considerable promise as general-purpose domain-agnostic approaches. However, it is not known whether the knowledge acquired on large-scale but potentially out-of-domain data can be effectively transferred across diverse scientific disciplines, and if a single, pretrained ViFM can be competitive with domain-specific baselines. To address this, we introduce SciVid, a comprehensive benchmark comprising five *Sci*entific *Vid*eo tasks, across medical computer vision, animal behavior, and weather forecasting. We adapt six leading ViFMs to SciVid using simple trainable readout modules, establishing strong baselines and demonstrating the potential for effective transfer learning. Specifically, we show that state-of-the-art results can be obtained in several applications by leveraging the general-purpose representations from ViFM backbones. Furthermore, our results reveal the limitations of existing ViFMs, and highlight opportunities for the development of generalizable models for high-impact scientific applications. We release our code at https://github.com/google-deepmind/scivid to facilitate further research in the development of ViFMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_03578 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications Hasson, Yana Luc, Pauline Momeni, Liliane Ovsjanikov, Maks Moing, Guillaume Le Kuznetsova, Alina Ktena, Ira Sun, Jennifer J. Koppula, Skanda Gokay, Dilara Heyward, Joseph Pot, Etienne Zisserman, Andrew Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning In recent years, there has been a proliferation of spatiotemporal foundation models in different scientific disciplines. While promising, these models are often domain-specific and are only assessed within the particular applications for which they are designed. Given that many tasks can be represented as video modeling problems, video foundation models (ViFMs) hold considerable promise as general-purpose domain-agnostic approaches. However, it is not known whether the knowledge acquired on large-scale but potentially out-of-domain data can be effectively transferred across diverse scientific disciplines, and if a single, pretrained ViFM can be competitive with domain-specific baselines. To address this, we introduce SciVid, a comprehensive benchmark comprising five *Sci*entific *Vid*eo tasks, across medical computer vision, animal behavior, and weather forecasting. We adapt six leading ViFMs to SciVid using simple trainable readout modules, establishing strong baselines and demonstrating the potential for effective transfer learning. Specifically, we show that state-of-the-art results can be obtained in several applications by leveraging the general-purpose representations from ViFM backbones. Furthermore, our results reveal the limitations of existing ViFMs, and highlight opportunities for the development of generalizable models for high-impact scientific applications. We release our code at https://github.com/google-deepmind/scivid to facilitate further research in the development of ViFMs. |
| title | SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.03578 |