Query matching for spatio-temporal action detection with query-based object detector
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913520864460800 |
|---|---|
| author | Hori, Shimon Omi, Kazuki Tamaki, Toru |
| author_facet | Hori, Shimon Omi, Kazuki Tamaki, Toru |
| contents | In this paper, we propose a method that extends the query-based object detection model, DETR, to spatio-temporal action detection, which requires maintaining temporal consistency in videos. Our proposed method applies DETR to each frame and uses feature shift to incorporate temporal information. However, DETR's object queries in each frame may correspond to different objects, making a simple feature shift ineffective. To overcome this issue, we propose query matching across different frames, ensuring that queries for the same object are matched and used for the feature shift. Experimental results show that performance on the JHMDB21 dataset improves significantly when query features are shifted using the proposed query matching. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_18408 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Query matching for spatio-temporal action detection with query-based object detector Hori, Shimon Omi, Kazuki Tamaki, Toru Computer Vision and Pattern Recognition In this paper, we propose a method that extends the query-based object detection model, DETR, to spatio-temporal action detection, which requires maintaining temporal consistency in videos. Our proposed method applies DETR to each frame and uses feature shift to incorporate temporal information. However, DETR's object queries in each frame may correspond to different objects, making a simple feature shift ineffective. To overcome this issue, we propose query matching across different frames, ensuring that queries for the same object are matched and used for the feature shift. Experimental results show that performance on the JHMDB21 dataset improves significantly when query features are shifted using the proposed query matching. |
| title | Query matching for spatio-temporal action detection with query-based object detector |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.18408 |