Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929225207906304 |
|---|---|
| author | Lin, Dan Lee, Philip Hann Yung Li, Yiming Wang, Ruoyu Yap, Kim-Hui Li, Bingbing Ngim, You Shing |
| author_facet | Lin, Dan Lee, Philip Hann Yung Li, Yiming Wang, Ruoyu Yap, Kim-Hui Li, Bingbing Ngim, You Shing |
| contents | Driver Action Recognition (DAR) is crucial in vehicle cabin monitoring systems. In real-world applications, it is common for vehicle cabins to be equipped with cameras featuring different modalities. However, multi-modality fusion strategies for the DAR task within car cabins have rarely been studied. In this paper, we propose a novel yet efficient multi-modality driver action recognition method based on dual feature shift, named DFS. DFS first integrates complementary features across modalities by performing modality feature interaction. Meanwhile, DFS achieves the neighbour feature propagation within single modalities, by feature shifting among temporal frames. To learn common patterns and improve model efficiency, DFS shares feature extracting stages among multiple modalities. Extensive experiments have been carried out to verify the effectiveness of the proposed DFS model on the Drive\&Act dataset. The results demonstrate that DFS achieves good performance and improves the efficiency of multi-modality driver action recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_14838 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring Lin, Dan Lee, Philip Hann Yung Li, Yiming Wang, Ruoyu Yap, Kim-Hui Li, Bingbing Ngim, You Shing Computer Vision and Pattern Recognition Driver Action Recognition (DAR) is crucial in vehicle cabin monitoring systems. In real-world applications, it is common for vehicle cabins to be equipped with cameras featuring different modalities. However, multi-modality fusion strategies for the DAR task within car cabins have rarely been studied. In this paper, we propose a novel yet efficient multi-modality driver action recognition method based on dual feature shift, named DFS. DFS first integrates complementary features across modalities by performing modality feature interaction. Meanwhile, DFS achieves the neighbour feature propagation within single modalities, by feature shifting among temporal frames. To learn common patterns and improve model efficiency, DFS shares feature extracting stages among multiple modalities. Extensive experiments have been carried out to verify the effectiveness of the proposed DFS model on the Drive\&Act dataset. The results demonstrate that DFS achieves good performance and improves the efficiency of multi-modality driver action recognition. |
| title | Multi-modality action recognition based on dual feature shift in vehicle cabin monitoring |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2401.14838 |