MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910947542564864 |
|---|---|
| author | Maldonado, Gabriel Pazho, Armin Danesh Noghre, Ghazal Alinezhad Katariya, Vinit Tabkhi, Hamed |
| author_facet | Maldonado, Gabriel Pazho, Armin Danesh Noghre, Ghazal Alinezhad Katariya, Vinit Tabkhi, Hamed |
| contents | Human motion generation is essential for fields such as animation, robotics, and virtual reality, requiring models that effectively capture motion dynamics from text descriptions. Existing approaches often rely on Contrastive Language-Image Pretraining (CLIP)-based text encoders, but their training on text-image pairs constrains their ability to understand temporal and kinematic structures inherent in motion and motion generation. This work introduces MoCLIP, a fine-tuned CLIP model with an additional motion encoding head, trained on motion sequences using contrastive learning and tethering loss. By explicitly incorporating motion-aware representations, MoCLIP enhances motion fidelity while remaining compatible with existing CLIP-based pipelines and seamlessly integrating into various CLIP-based methods. Experiments demonstrate that MoCLIP improves Top-1, Top-2, and Top-3 accuracy while maintaining competitive FID, leading to improved text-to-motion alignment results. These results highlight MoCLIP's versatility and effectiveness, establishing it as a robust framework for enhancing motion generation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_10810 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation Maldonado, Gabriel Pazho, Armin Danesh Noghre, Ghazal Alinezhad Katariya, Vinit Tabkhi, Hamed Computer Vision and Pattern Recognition Human motion generation is essential for fields such as animation, robotics, and virtual reality, requiring models that effectively capture motion dynamics from text descriptions. Existing approaches often rely on Contrastive Language-Image Pretraining (CLIP)-based text encoders, but their training on text-image pairs constrains their ability to understand temporal and kinematic structures inherent in motion and motion generation. This work introduces MoCLIP, a fine-tuned CLIP model with an additional motion encoding head, trained on motion sequences using contrastive learning and tethering loss. By explicitly incorporating motion-aware representations, MoCLIP enhances motion fidelity while remaining compatible with existing CLIP-based pipelines and seamlessly integrating into various CLIP-based methods. Experiments demonstrate that MoCLIP improves Top-1, Top-2, and Top-3 accuracy while maintaining competitive FID, leading to improved text-to-motion alignment results. These results highlight MoCLIP's versatility and effectiveness, establishing it as a robust framework for enhancing motion generation. |
| title | MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.10810 |