A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917546170515456 |
|---|---|
| author | Fateh, Fawad Javed Ali, Ali Shah Popattia, Murad Nizamani, Usman Konin, Andrey Zia, M. Zeeshan Tran, Quoc-Huy |
| author_facet | Fateh, Fawad Javed Ali, Ali Shah Popattia, Murad Nizamani, Usman Konin, Andrey Zia, M. Zeeshan Tran, Quoc-Huy |
| contents | We present a novel hierarchical spatiotemporal action tokenizer for in-context imitation learning. We first propose a hierarchical approach, which consists of two successive levels of vector quantization. In particular, the lower level assigns input actions to fine-grained subclusters, while the higher level further maps fine-grained subclusters to clusters. Our hierarchical approach outperforms the non-hierarchical counterpart, while mainly exploiting spatial information by reconstructing input actions. Furthermore, we extend our approach by utilizing both spatial and temporal cues, forming a hierarchical spatiotemporal action tokenizer, namely HiST-AT. Specifically, our hierarchical spatiotemporal approach conducts multi-level clustering, while simultaneously recovering input actions and their associated timestamps. Finally, extensive evaluations on multiple simulation and real robotic manipulation benchmarks show that our approach establishes a new state-of-the-art performance in in-context imitation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_15215 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics Fateh, Fawad Javed Ali, Ali Shah Popattia, Murad Nizamani, Usman Konin, Andrey Zia, M. Zeeshan Tran, Quoc-Huy Robotics We present a novel hierarchical spatiotemporal action tokenizer for in-context imitation learning. We first propose a hierarchical approach, which consists of two successive levels of vector quantization. In particular, the lower level assigns input actions to fine-grained subclusters, while the higher level further maps fine-grained subclusters to clusters. Our hierarchical approach outperforms the non-hierarchical counterpart, while mainly exploiting spatial information by reconstructing input actions. Furthermore, we extend our approach by utilizing both spatial and temporal cues, forming a hierarchical spatiotemporal action tokenizer, namely HiST-AT. Specifically, our hierarchical spatiotemporal approach conducts multi-level clustering, while simultaneously recovering input actions and their associated timestamps. Finally, extensive evaluations on multiple simulation and real robotic manipulation benchmarks show that our approach establishes a new state-of-the-art performance in in-context imitation learning. |
| title | A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Robotics |
| topic | Robotics |
| url | https://arxiv.org/abs/2604.15215 |