Background-aware Moment Detection for Video Moment Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914961566990336 |
|---|---|
| author | Jung, Minjoon Jang, Youwon Choi, Seongho Kim, Joochan Kim, Jin-Hwa Zhang, Byoung-Tak |
| author_facet | Jung, Minjoon Jang, Youwon Choi, Seongho Kim, Joochan Kim, Jin-Hwa Zhang, Byoung-Tak |
| contents | Video moment retrieval (VMR) identifies a specific moment in an untrimmed video for a given natural language query. This task is prone to suffer the weak alignment problem innate in video datasets. Due to the ambiguity, a query does not fully cover the relevant details of the corresponding moment, or the moment may contain misaligned and irrelevant frames, potentially limiting further performance gains. To tackle this problem, we propose a background-aware moment detection transformer (BM-DETR). Our model adopts a contrastive approach, carefully utilizing the negative queries matched to other moments in the video. Specifically, our model learns to predict the target moment from the joint probability of each frame given the positive query and the complement of negative queries. This leads to effective use of the surrounding background, improving moment sensitivity and enhancing overall alignments in videos. Extensive experiments on four benchmarks demonstrate the effectiveness of our approach. Our code is available at: \url{https://github.com/minjoong507/BM-DETR} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_02728 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Background-aware Moment Detection for Video Moment Retrieval Jung, Minjoon Jang, Youwon Choi, Seongho Kim, Joochan Kim, Jin-Hwa Zhang, Byoung-Tak Computer Vision and Pattern Recognition Video moment retrieval (VMR) identifies a specific moment in an untrimmed video for a given natural language query. This task is prone to suffer the weak alignment problem innate in video datasets. Due to the ambiguity, a query does not fully cover the relevant details of the corresponding moment, or the moment may contain misaligned and irrelevant frames, potentially limiting further performance gains. To tackle this problem, we propose a background-aware moment detection transformer (BM-DETR). Our model adopts a contrastive approach, carefully utilizing the negative queries matched to other moments in the video. Specifically, our model learns to predict the target moment from the joint probability of each frame given the positive query and the complement of negative queries. This leads to effective use of the surrounding background, improving moment sensitivity and enhancing overall alignments in videos. Extensive experiments on four benchmarks demonstrate the effectiveness of our approach. Our code is available at: \url{https://github.com/minjoong507/BM-DETR} |
| title | Background-aware Moment Detection for Video Moment Retrieval |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2306.02728 |