Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908139010392064 |
|---|---|
| author | Seliunina, Svetlana Otelepko, Artem Memmesheimer, Raphael Behnke, Sven |
| author_facet | Seliunina, Svetlana Otelepko, Artem Memmesheimer, Raphael Behnke, Sven |
| contents | Robots need to perceive persons in their surroundings for safety and to interact with them. In this paper, we present a person segmentation and action classification approach that operates on 3D scans of hemisphere field of view LiDAR sensors. We recorded a data set with an Ouster OSDome-64 sensor consisting of scenes where persons perform three different actions and annotated it. We propose a method based on a MaskDINO model to detect and segment persons and to recognize their actions from combined spherical projected multi-channel representations of the LiDAR data with an additional positional encoding. Our approach demonstrates good performance for the person segmentation task and further performs well for the estimation of the person action states walking, waving, and sitting. An ablation study provides insights about the individual channel contributions for the person segmentation task. The trained models, code and dataset are made publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11151 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors Seliunina, Svetlana Otelepko, Artem Memmesheimer, Raphael Behnke, Sven Computer Vision and Pattern Recognition Robotics Robots need to perceive persons in their surroundings for safety and to interact with them. In this paper, we present a person segmentation and action classification approach that operates on 3D scans of hemisphere field of view LiDAR sensors. We recorded a data set with an Ouster OSDome-64 sensor consisting of scenes where persons perform three different actions and annotated it. We propose a method based on a MaskDINO model to detect and segment persons and to recognize their actions from combined spherical projected multi-channel representations of the LiDAR data with an additional positional encoding. Our approach demonstrates good performance for the person segmentation task and further performs well for the estimation of the person action states walking, waving, and sitting. An ablation study provides insights about the individual channel contributions for the person segmentation task. The trained models, code and dataset are made publicly available. |
| title | Person Segmentation and Action Classification for Multi-Channel Hemisphere Field of View LiDAR Sensors |
| topic | Computer Vision and Pattern Recognition Robotics |
| url | https://arxiv.org/abs/2411.11151 |