STF: Spatio-Temporal Fusion Module for Improving Video Object Detection
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909109276639232 |
|---|---|
| author | Anwar, Noreen Bilodeau, Guillaume-Alexandre Bouachir, Wassim |
| author_facet | Anwar, Noreen Bilodeau, Guillaume-Alexandre Bouachir, Wassim |
| contents | Consecutive frames in a video contain redundancy, but they may also contain relevant complementary information for the detection task. The objective of our work is to leverage this complementary information to improve detection. Therefore, we propose a spatio-temporal fusion framework (STF). We first introduce multi-frame and single-frame attention modules that allow a neural network to share feature maps between nearby frames to obtain more robust object representations. Second, we introduce a dual-frame fusion module that merges feature maps in a learnable manner to improve them. Our evaluation is conducted on three different benchmarks including video sequences of moving road users. The performed experiments demonstrate that the proposed spatio-temporal fusion module leads to improved detection performance compared to baseline object detectors. Code is available at https://github.com/noreenanwar/STF-module |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10752 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | STF: Spatio-Temporal Fusion Module for Improving Video Object Detection Anwar, Noreen Bilodeau, Guillaume-Alexandre Bouachir, Wassim Computer Vision and Pattern Recognition Consecutive frames in a video contain redundancy, but they may also contain relevant complementary information for the detection task. The objective of our work is to leverage this complementary information to improve detection. Therefore, we propose a spatio-temporal fusion framework (STF). We first introduce multi-frame and single-frame attention modules that allow a neural network to share feature maps between nearby frames to obtain more robust object representations. Second, we introduce a dual-frame fusion module that merges feature maps in a learnable manner to improve them. Our evaluation is conducted on three different benchmarks including video sequences of moving road users. The performed experiments demonstrate that the proposed spatio-temporal fusion module leads to improved detection performance compared to baseline object detectors. Code is available at https://github.com/noreenanwar/STF-module |
| title | STF: Spatio-Temporal Fusion Module for Improving Video Object Detection |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.10752 |