Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitching
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915181195427840 |
|---|---|
| author | Ricciardi, Antonio Pio Maiorca, Valentino Moschella, Luca Marin, Riccardo Rodolà, Emanuele |
| author_facet | Ricciardi, Antonio Pio Maiorca, Valentino Moschella, Luca Marin, Riccardo Rodolà, Emanuele |
| contents | Deep Reinforcement Learning (RL) models often fail to generalize when even small changes occur in the environment's observations or task requirements. Addressing these shifts typically requires costly retraining, limiting the reusability of learned policies. In this paper, we build on recent work in semantic alignment to propose a zero-shot method for mapping between latent spaces across different agents trained on different visual and task variations. Specifically, we learn a transformation that maps embeddings from one agent's encoder to another agent's encoder without further fine-tuning. Our approach relies on a small set of "anchor" observations that are semantically aligned, which we use to estimate an affine or orthogonal transform. Once the transformation is found, an existing controller trained for one domain can interpret embeddings from a different (existing) encoder in a zero-shot fashion, skipping additional trainings. We empirically demonstrate that our framework preserves high performance under visual and task domain shifts. We empirically demonstrate zero-shot stitching performance on the CarRacing environment with changing background and task. By allowing modular re-assembly of existing policies, it paves the way for more robust, compositional RL in dynamically changing environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01881 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitching Ricciardi, Antonio Pio Maiorca, Valentino Moschella, Luca Marin, Riccardo Rodolà, Emanuele Machine Learning Artificial Intelligence 68T07 I.2.6 Deep Reinforcement Learning (RL) models often fail to generalize when even small changes occur in the environment's observations or task requirements. Addressing these shifts typically requires costly retraining, limiting the reusability of learned policies. In this paper, we build on recent work in semantic alignment to propose a zero-shot method for mapping between latent spaces across different agents trained on different visual and task variations. Specifically, we learn a transformation that maps embeddings from one agent's encoder to another agent's encoder without further fine-tuning. Our approach relies on a small set of "anchor" observations that are semantically aligned, which we use to estimate an affine or orthogonal transform. Once the transformation is found, an existing controller trained for one domain can interpret embeddings from a different (existing) encoder in a zero-shot fashion, skipping additional trainings. We empirically demonstrate that our framework preserves high performance under visual and task domain shifts. We empirically demonstrate zero-shot stitching performance on the CarRacing environment with changing background and task. By allowing modular re-assembly of existing policies, it paves the way for more robust, compositional RL in dynamically changing environments. |
| title | Mapping representations in Reinforcement Learning via Semantic Alignment for Zero-Shot Stitching |
| topic | Machine Learning Artificial Intelligence 68T07 I.2.6 |
| url | https://arxiv.org/abs/2503.01881 |