FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913218817949696 |
|---|---|
| author | Yagi, Takuma Ohashi, Misaki Huang, Yifei Furuta, Ryosuke Adachi, Shungo Mitsuyama, Toutai Sato, Yoichi |
| author_facet | Yagi, Takuma Ohashi, Misaki Huang, Yifei Furuta, Ryosuke Adachi, Shungo Mitsuyama, Toutai Sato, Yoichi |
| contents | In the development of science, accurate and reproducible documentation of the experimental process is crucial. Automatic recognition of the actions in experiments from videos would help experimenters by complementing the recording of experiments. Towards this goal, we propose FineBio, a new fine-grained video dataset of people performing biological experiments. The dataset consists of multi-view videos of 32 participants performing mock biological experiments with a total duration of 14.5 hours. One experiment forms a hierarchical structure, where a protocol consists of several steps, each further decomposed into a set of atomic operations. The uniqueness of biological experiments is that while they require strict adherence to steps described in each protocol, there is freedom in the order of atomic operations. We provide hierarchical annotation on protocols, steps, atomic operations, object locations, and their manipulation states, providing new challenges for structured activity understanding and hand-object interaction recognition. To find out challenges on activity understanding in biological experiments, we introduce baseline models and results on four different tasks, including (i) step segmentation, (ii) atomic operation detection (iii) object detection, and (iv) manipulated/affected object detection. Dataset and code are available from https://github.com/aistairc/FineBio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00293 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation Yagi, Takuma Ohashi, Misaki Huang, Yifei Furuta, Ryosuke Adachi, Shungo Mitsuyama, Toutai Sato, Yoichi Computer Vision and Pattern Recognition In the development of science, accurate and reproducible documentation of the experimental process is crucial. Automatic recognition of the actions in experiments from videos would help experimenters by complementing the recording of experiments. Towards this goal, we propose FineBio, a new fine-grained video dataset of people performing biological experiments. The dataset consists of multi-view videos of 32 participants performing mock biological experiments with a total duration of 14.5 hours. One experiment forms a hierarchical structure, where a protocol consists of several steps, each further decomposed into a set of atomic operations. The uniqueness of biological experiments is that while they require strict adherence to steps described in each protocol, there is freedom in the order of atomic operations. We provide hierarchical annotation on protocols, steps, atomic operations, object locations, and their manipulation states, providing new challenges for structured activity understanding and hand-object interaction recognition. To find out challenges on activity understanding in biological experiments, we introduce baseline models and results on four different tasks, including (i) step segmentation, (ii) atomic operation detection (iii) object detection, and (iv) manipulated/affected object detection. Dataset and code are available from https://github.com/aistairc/FineBio. |
| title | FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.00293 |