Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908976198713344 |
|---|---|
| author | Wang, Yiming Li, Frederick W. B. Wang, Jingyun |
| author_facet | Wang, Yiming Li, Frederick W. B. Wang, Jingyun |
| contents | Zero-shot action recognition is challenging due to the semantic gap between seen and unseen classes. We present a novel framework that enhances CLIP with disentangled embeddings and semantic-guided interaction. A Motion Separation Module (MSM) separates motion-sensitive and global-static features, while a Motion Aggregation Block (MAB) employs gated cross-attention to refine motion representation without re-coupling redundant information. To facilitate generalization to unseen categories, we enforce semantic alignment between video features and textual representations by aligning projected embeddings with positive textual prompts, while leveraging negative prompts to explicitly model "non-class" semantics. Experiments on standard benchmarks demonstrate that our method consistently outperforms prior CLIP-based approaches, achieving robust zero-shot action recognition across both coarse and fine-grained datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_17062 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition Wang, Yiming Li, Frederick W. B. Wang, Jingyun Computer Vision and Pattern Recognition I.2.10; I.4.8; I.5.4 Zero-shot action recognition is challenging due to the semantic gap between seen and unseen classes. We present a novel framework that enhances CLIP with disentangled embeddings and semantic-guided interaction. A Motion Separation Module (MSM) separates motion-sensitive and global-static features, while a Motion Aggregation Block (MAB) employs gated cross-attention to refine motion representation without re-coupling redundant information. To facilitate generalization to unseen categories, we enforce semantic alignment between video features and textual representations by aligning projected embeddings with positive textual prompts, while leveraging negative prompts to explicitly model "non-class" semantics. Experiments on standard benchmarks demonstrate that our method consistently outperforms prior CLIP-based approaches, achieving robust zero-shot action recognition across both coarse and fine-grained datasets. |
| title | Motion-Guided Semantic Alignment with Negative Prompts for Zero-Shot Video Action Recognition |
| topic | Computer Vision and Pattern Recognition I.2.10; I.4.8; I.5.4 |
| url | https://arxiv.org/abs/2604.17062 |