Text me the data: Generating Ground Pressure Sequence from Textual Descriptions for HAR
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929252676403200 |
|---|---|
| author | Ray, Lala Shakti Swarup Zhou, Bo Suh, Sungho Krupp, Lars Rey, Vitor Fortes Lukowicz, Paul |
| author_facet | Ray, Lala Shakti Swarup Zhou, Bo Suh, Sungho Krupp, Lars Rey, Vitor Fortes Lukowicz, Paul |
| contents | In human activity recognition (HAR), the availability of substantial ground truth is necessary for training efficient models. However, acquiring ground pressure data through physical sensors itself can be cost-prohibitive, time-consuming. To address this critical need, we introduce Text-to-Pressure (T2P), a framework designed to generate extensive ground pressure sequences from textual descriptions of human activities using deep learning techniques. We show that the combination of vector quantization of sensor data along with simple text conditioned auto regressive strategy allows us to obtain high-quality generated pressure sequences from textual descriptions with the help of discrete latent correlation between text and pressure maps. We achieved comparable performance on the consistency between text and generated motion with an R squared value of 0.722, Masked R squared value of 0.892, and FID score of 1.83. Additionally, we trained a HAR model with the the synthesized data and evaluated it on pressure dynamics collected by a real pressure sensor which is on par with a model trained on only real data. Combining both real and synthesized training data increases the overall macro F1 score by 5.9 percent. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_14427 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Text me the data: Generating Ground Pressure Sequence from Textual Descriptions for HAR Ray, Lala Shakti Swarup Zhou, Bo Suh, Sungho Krupp, Lars Rey, Vitor Fortes Lukowicz, Paul Machine Learning Computation and Language Signal Processing In human activity recognition (HAR), the availability of substantial ground truth is necessary for training efficient models. However, acquiring ground pressure data through physical sensors itself can be cost-prohibitive, time-consuming. To address this critical need, we introduce Text-to-Pressure (T2P), a framework designed to generate extensive ground pressure sequences from textual descriptions of human activities using deep learning techniques. We show that the combination of vector quantization of sensor data along with simple text conditioned auto regressive strategy allows us to obtain high-quality generated pressure sequences from textual descriptions with the help of discrete latent correlation between text and pressure maps. We achieved comparable performance on the consistency between text and generated motion with an R squared value of 0.722, Masked R squared value of 0.892, and FID score of 1.83. Additionally, we trained a HAR model with the the synthesized data and evaluated it on pressure dynamics collected by a real pressure sensor which is on par with a model trained on only real data. Combining both real and synthesized training data increases the overall macro F1 score by 5.9 percent. |
| title | Text me the data: Generating Ground Pressure Sequence from Textual Descriptions for HAR |
| topic | Machine Learning Computation and Language Signal Processing |
| url | https://arxiv.org/abs/2402.14427 |