CigTime: Corrective Instruction Generation Through Inverse Motion Editing
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917861041111040 |
|---|---|
| author | Fang, Qihang Tang, Chengcheng Tekin, Bugra Yang, Yanchao |
| author_facet | Fang, Qihang Tang, Chengcheng Tekin, Bugra Yang, Yanchao |
| contents | Recent advancements in models linking natural language with human motions have shown significant promise in motion generation and editing based on instructional text. Motivated by applications in sports coaching and motor skill learning, we investigate the inverse problem: generating corrective instructional text, leveraging motion editing and generation models. We introduce a novel approach that, given a user's current motion (source) and the desired motion (target), generates text instructions to guide the user towards achieving the target motion. We leverage large language models to generate corrective texts and utilize existing motion generation and editing frameworks to compile datasets of triplets (source motion, target motion, and corrective text). Using this data, we propose a new motion-language model for generating corrective instructions. We present both qualitative and quantitative results across a diverse range of applications that largely improve upon baselines. Our approach demonstrates its effectiveness in instructional scenarios, offering text-based guidance to correct and enhance user performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_05460 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CigTime: Corrective Instruction Generation Through Inverse Motion Editing Fang, Qihang Tang, Chengcheng Tekin, Bugra Yang, Yanchao Computer Vision and Pattern Recognition 68T45 I.4.9 Recent advancements in models linking natural language with human motions have shown significant promise in motion generation and editing based on instructional text. Motivated by applications in sports coaching and motor skill learning, we investigate the inverse problem: generating corrective instructional text, leveraging motion editing and generation models. We introduce a novel approach that, given a user's current motion (source) and the desired motion (target), generates text instructions to guide the user towards achieving the target motion. We leverage large language models to generate corrective texts and utilize existing motion generation and editing frameworks to compile datasets of triplets (source motion, target motion, and corrective text). Using this data, we propose a new motion-language model for generating corrective instructions. We present both qualitative and quantitative results across a diverse range of applications that largely improve upon baselines. Our approach demonstrates its effectiveness in instructional scenarios, offering text-based guidance to correct and enhance user performance. |
| title | CigTime: Corrective Instruction Generation Through Inverse Motion Editing |
| topic | Computer Vision and Pattern Recognition 68T45 I.4.9 |
| url | https://arxiv.org/abs/2412.05460 |