Resource-Efficient Multiview Perception: Integrating Semantic Masking with Masked Autoencoders
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915604880949248 |
|---|---|
| author | Dakic, Kosta Thilakarathna, Kanchana Calheiros, Rodrigo N. Lim, Teng Joon |
| author_facet | Dakic, Kosta Thilakarathna, Kanchana Calheiros, Rodrigo N. Lim, Teng Joon |
| contents | Multiview systems have become a key technology in modern computer vision, offering advanced capabilities in scene understanding and analysis. However, these systems face critical challenges in bandwidth limitations and computational constraints, particularly for resource-limited camera nodes like drones. This paper presents a novel approach for communication-efficient distributed multiview detection and tracking using masked autoencoders (MAEs). We introduce a semantic-guided masking strategy that leverages pre-trained segmentation models and a tunable power function to prioritize informative image regions. This approach, combined with an MAE, reduces communication overhead while preserving essential visual information. We evaluate our method on both virtual and real-world multiview datasets, demonstrating comparable performance in terms of detection and tracking performance metrics compared to state-of-the-art techniques, even at high masking ratios. Our selective masking algorithm outperforms random masking, maintaining higher accuracy and precision as the masking ratio increases. Furthermore, our approach achieves a significant reduction in transmission data volume compared to baseline methods, thereby balancing multiview tracking performance with communication efficiency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_04817 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Resource-Efficient Multiview Perception: Integrating Semantic Masking with Masked Autoencoders Dakic, Kosta Thilakarathna, Kanchana Calheiros, Rodrigo N. Lim, Teng Joon Computer Vision and Pattern Recognition Artificial Intelligence Image and Video Processing Signal Processing Multiview systems have become a key technology in modern computer vision, offering advanced capabilities in scene understanding and analysis. However, these systems face critical challenges in bandwidth limitations and computational constraints, particularly for resource-limited camera nodes like drones. This paper presents a novel approach for communication-efficient distributed multiview detection and tracking using masked autoencoders (MAEs). We introduce a semantic-guided masking strategy that leverages pre-trained segmentation models and a tunable power function to prioritize informative image regions. This approach, combined with an MAE, reduces communication overhead while preserving essential visual information. We evaluate our method on both virtual and real-world multiview datasets, demonstrating comparable performance in terms of detection and tracking performance metrics compared to state-of-the-art techniques, even at high masking ratios. Our selective masking algorithm outperforms random masking, maintaining higher accuracy and precision as the masking ratio increases. Furthermore, our approach achieves a significant reduction in transmission data volume compared to baseline methods, thereby balancing multiview tracking performance with communication efficiency. |
| title | Resource-Efficient Multiview Perception: Integrating Semantic Masking with Masked Autoencoders |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Image and Video Processing Signal Processing |
| url | https://arxiv.org/abs/2410.04817 |