ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910546292375552 |
|---|---|
| author | Martin-Turrero, Carmen Bouvier, Maxence Breitenstein, Manuel Zanuttigh, Pietro Parret, Vincent |
| author_facet | Martin-Turrero, Carmen Bouvier, Maxence Breitenstein, Manuel Zanuttigh, Pietro Parret, Vincent |
| contents | We seek to enable classic processing of continuous ultra-sparse spatiotemporal data generated by event-based sensors with dense machine learning models. We propose a novel hybrid pipeline composed of asynchronous sensing and synchronous processing that combines several ideas: (1) an embedding based on PointNet models -- the ALERT module -- that can continuously integrate new and dismiss old events thanks to a leakage mechanism, (2) a flexible readout of the embedded data that allows to feed any downstream model with always up-to-date features at any sampling rate, (3) exploiting the input sparsity in a patch-based approach inspired by Vision Transformer to optimize the efficiency of the method. These embeddings are then processed by a transformer model trained for object and gesture recognition. Using this approach, we achieve performances at the state-of-the-art with a lower latency than competitors. We also demonstrate that our asynchronous model can operate at any desired sampling rate. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01393 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data Martin-Turrero, Carmen Bouvier, Maxence Breitenstein, Manuel Zanuttigh, Pietro Parret, Vincent Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing 68T05 I.2.6; I.2.10; I.4.8; I.4.10; D.2.2; D.1.4 We seek to enable classic processing of continuous ultra-sparse spatiotemporal data generated by event-based sensors with dense machine learning models. We propose a novel hybrid pipeline composed of asynchronous sensing and synchronous processing that combines several ideas: (1) an embedding based on PointNet models -- the ALERT module -- that can continuously integrate new and dismiss old events thanks to a leakage mechanism, (2) a flexible readout of the embedded data that allows to feed any downstream model with always up-to-date features at any sampling rate, (3) exploiting the input sparsity in a patch-based approach inspired by Vision Transformer to optimize the efficiency of the method. These embeddings are then processed by a transformer model trained for object and gesture recognition. Using this approach, we achieve performances at the state-of-the-art with a lower latency than competitors. We also demonstrate that our asynchronous model can operate at any desired sampling rate. |
| title | ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data |
| topic | Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing 68T05 I.2.6; I.2.10; I.4.8; I.4.10; D.2.2; D.1.4 |
| url | https://arxiv.org/abs/2402.01393 |