Motion-Based Sign Language Video Summarization using Curvature and Torsion
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909365741551616 |
|---|---|
| author | Sartinas, Evangelos G. Psarakis, Emmanouil Z. Kosmopoulos, Dimitrios I. |
| author_facet | Sartinas, Evangelos G. Psarakis, Emmanouil Z. Kosmopoulos, Dimitrios I. |
| contents | An interesting problem in many video-based applications is the generation of short synopses by selecting the most informative frames, a procedure which is known as video summarization. For sign language videos the benefits of using the $t$-parameterized counterpart of the curvature of the 2-D signer's wrist trajectory to identify keyframes, have been recently reported in the literature. In this paper we extend these ideas by modeling the 3-D hand motion that is extracted from each frame of the video. To this end we propose a new informative function based on the $t$-parameterized curvature and torsion of the 3-D trajectory. The method to characterize video frames as keyframes depends on whether the motion occurs in 2-D or 3-D space. Specifically, in the case of 3-D motion we look for the maxima of the harmonic mean of the curvature and torsion of the target's trajectory; in the planar motion case we seek for the maxima of the trajectory's curvature. The proposed 3-D feature is experimentally evaluated in applications of sign language videos on (1) objective measures using ground-truth keyframe annotations, (2) human-based evaluation of understanding, and (3) gloss classification and the results obtained are promising. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_16801 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Motion-Based Sign Language Video Summarization using Curvature and Torsion Sartinas, Evangelos G. Psarakis, Emmanouil Z. Kosmopoulos, Dimitrios I. Computer Vision and Pattern Recognition Computation and Language 68T45, 68U10 I.4.9; I.5.4; I.2.7 An interesting problem in many video-based applications is the generation of short synopses by selecting the most informative frames, a procedure which is known as video summarization. For sign language videos the benefits of using the $t$-parameterized counterpart of the curvature of the 2-D signer's wrist trajectory to identify keyframes, have been recently reported in the literature. In this paper we extend these ideas by modeling the 3-D hand motion that is extracted from each frame of the video. To this end we propose a new informative function based on the $t$-parameterized curvature and torsion of the 3-D trajectory. The method to characterize video frames as keyframes depends on whether the motion occurs in 2-D or 3-D space. Specifically, in the case of 3-D motion we look for the maxima of the harmonic mean of the curvature and torsion of the target's trajectory; in the planar motion case we seek for the maxima of the trajectory's curvature. The proposed 3-D feature is experimentally evaluated in applications of sign language videos on (1) objective measures using ground-truth keyframe annotations, (2) human-based evaluation of understanding, and (3) gloss classification and the results obtained are promising. |
| title | Motion-Based Sign Language Video Summarization using Curvature and Torsion |
| topic | Computer Vision and Pattern Recognition Computation and Language 68T45, 68U10 I.4.9; I.5.4; I.2.7 |
| url | https://arxiv.org/abs/2305.16801 |