Quantization-Free Autoregressive Action Transformer
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911307408605184 |
|---|---|
| author | Sheebaelhamd, Ziyad Tschannen, Michael Muehlebach, Michael Vernade, Claire |
| author_facet | Sheebaelhamd, Ziyad Tschannen, Michael Muehlebach, Michael Vernade, Claire |
| contents | Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_14259 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Quantization-Free Autoregressive Action Transformer Sheebaelhamd, Ziyad Tschannen, Michael Muehlebach, Michael Vernade, Claire Machine Learning Robotics Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results. |
| title | Quantization-Free Autoregressive Action Transformer |
| topic | Machine Learning Robotics |
| url | https://arxiv.org/abs/2503.14259 |