Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2408.12981 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916367588917248 |
|---|---|
| author | Gao, Chenghua Li, Min Liu, Jianshuo Ren, Junxing Chen, Lin Liu, Haoyu Meng, Bo Fu, Jitao Su, Wenwen |
| author_facet | Gao, Chenghua Li, Min Liu, Jianshuo Ren, Junxing Chen, Lin Liu, Haoyu Meng, Bo Fu, Jitao Su, Wenwen |
| contents | Video Moment Retrieval (VMR) aims to retrieve relevant moments of an untrimmed video corresponding to the query. While cross-modal interaction approaches have shown progress in filtering out query-irrelevant information in videos, they assume the precise alignment between the query semantics and the corresponding video moments, potentially overlooking the misunderstanding of the natural language semantics. To address this challenge, we propose a novel model called \textit{QD-VMR}, a query debiasing model with enhanced contextual understanding. Firstly, we leverage a Global Partial Aligner module via video clip and query features alignment and video-query contrastive learning to enhance the cross-modal understanding capabilities of the model. Subsequently, we employ a Query Debiasing Module to obtain debiased query features efficiently, and a Visual Enhancement module to refine the video features related to the query. Finally, we adopt the DETR structure to predict the possible target video moments. Through extensive evaluations of three benchmark datasets, QD-VMR achieves state-of-the-art performance, proving its potential to improve the accuracy of VMR. Further analytical experiments demonstrate the effectiveness of our proposed module. Our code will be released to facilitate future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_12981 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval Gao, Chenghua Li, Min Liu, Jianshuo Ren, Junxing Chen, Lin Liu, Haoyu Meng, Bo Fu, Jitao Su, Wenwen Artificial Intelligence Video Moment Retrieval (VMR) aims to retrieve relevant moments of an untrimmed video corresponding to the query. While cross-modal interaction approaches have shown progress in filtering out query-irrelevant information in videos, they assume the precise alignment between the query semantics and the corresponding video moments, potentially overlooking the misunderstanding of the natural language semantics. To address this challenge, we propose a novel model called \textit{QD-VMR}, a query debiasing model with enhanced contextual understanding. Firstly, we leverage a Global Partial Aligner module via video clip and query features alignment and video-query contrastive learning to enhance the cross-modal understanding capabilities of the model. Subsequently, we employ a Query Debiasing Module to obtain debiased query features efficiently, and a Visual Enhancement module to refine the video features related to the query. Finally, we adopt the DETR structure to predict the possible target video moments. Through extensive evaluations of three benchmark datasets, QD-VMR achieves state-of-the-art performance, proving its potential to improve the accuracy of VMR. Further analytical experiments demonstrate the effectiveness of our proposed module. Our code will be released to facilitate future research. |
| title | QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2408.12981 |