Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918125620953088 |
|---|---|
| author | Liu, Weijia Cao, Jiuxin Miao, Bo Fu, Zhiheng Zhu, Xuelin Ge, Jiawei Liu, Bo Nasim, Mehwish Mian, Ajmal |
| author_facet | Liu, Weijia Cao, Jiuxin Miao, Bo Fu, Zhiheng Zhu, Xuelin Ge, Jiawei Liu, Bo Nasim, Mehwish Mian, Ajmal |
| contents | Current text-driven Video Moment Retrieval (VMR) methods encode all video clips, including irrelevant ones, disrupting multimodal alignment and hindering optimization. To this end, we propose a denoise-then-retrieve paradigm that explicitly filters text-irrelevant clips from videos and then retrieves the target moment using purified multimodal representations. Following this paradigm, we introduce the Denoise-then-Retrieve Network (DRNet), comprising Text-Conditioned Denoising (TCD) and Text-Reconstruction Feedback (TRF) modules. TCD integrates cross-attention and structured state space blocks to dynamically identify noisy clips and produce a noise mask to purify multimodal video representations. TRF further distills a single query embedding from purified video representations and aligns it with the text embedding, serving as auxiliary supervision for denoising during training. Finally, we perform conditional retrieval using text embeddings on purified video representations for accurate VMR. Experiments on Charades-STA and QVHighlights demonstrate that our approach surpasses state-of-the-art methods on all metrics. Furthermore, our denoise-then-retrieve paradigm is adaptable and can be seamlessly integrated into advanced VMR models to boost performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_11313 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval Liu, Weijia Cao, Jiuxin Miao, Bo Fu, Zhiheng Zhu, Xuelin Ge, Jiawei Liu, Bo Nasim, Mehwish Mian, Ajmal Computer Vision and Pattern Recognition Current text-driven Video Moment Retrieval (VMR) methods encode all video clips, including irrelevant ones, disrupting multimodal alignment and hindering optimization. To this end, we propose a denoise-then-retrieve paradigm that explicitly filters text-irrelevant clips from videos and then retrieves the target moment using purified multimodal representations. Following this paradigm, we introduce the Denoise-then-Retrieve Network (DRNet), comprising Text-Conditioned Denoising (TCD) and Text-Reconstruction Feedback (TRF) modules. TCD integrates cross-attention and structured state space blocks to dynamically identify noisy clips and produce a noise mask to purify multimodal video representations. TRF further distills a single query embedding from purified video representations and aligns it with the text embedding, serving as auxiliary supervision for denoising during training. Finally, we perform conditional retrieval using text embeddings on purified video representations for accurate VMR. Experiments on Charades-STA and QVHighlights demonstrate that our approach surpasses state-of-the-art methods on all metrics. Furthermore, our denoise-then-retrieve paradigm is adaptable and can be seamlessly integrated into advanced VMR models to boost performance. |
| title | Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.11313 |