Learning Additively Compositional Latent Actions for Embodied AI
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914444933595136 |
|---|---|
| author | Wei, Hangxing Chen, Xiaoyu Zhang, Chuheng Pearce, Tim Chen, Jianyu Lamb, Alex Zhao, Li Bian, Jiang |
| author_facet | Wei, Hangxing Chen, Xiaoyu Zhang, Chuheng Pearce, Tim Chen, Jianyu Lamb, Alex Zhao, Li Bian, Jiang |
| contents | Latent action learning infers pseudo-action labels from visual transitions, providing an approach to leverage internet-scale video for embodied AI. However, most methods learn latent actions without structural priors that encode the additive, compositional structure of physical motion. As a result, latents often entangle irrelevant scene details or information about future observations with true state changes and miscalibrate motion magnitude. We introduce Additively Compositional Latent Action Model (AC-LAM), which enforces scene-wise additive composition structure over short horizons on the latent action space. These AC constraints encourage simple algebraic structure in the latent action space~(identity, inverse, cycle consistency) and suppress information that does not compose additively. Empirically, AC-LAM learns more structured, motion-specific, and displacement-calibrated latent actions and provides stronger supervision for downstream policy learning, outperforming state-of-the-art LAMs across simulated and real-world tabletop tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_03340 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learning Additively Compositional Latent Actions for Embodied AI Wei, Hangxing Chen, Xiaoyu Zhang, Chuheng Pearce, Tim Chen, Jianyu Lamb, Alex Zhao, Li Bian, Jiang Computer Vision and Pattern Recognition Artificial Intelligence Latent action learning infers pseudo-action labels from visual transitions, providing an approach to leverage internet-scale video for embodied AI. However, most methods learn latent actions without structural priors that encode the additive, compositional structure of physical motion. As a result, latents often entangle irrelevant scene details or information about future observations with true state changes and miscalibrate motion magnitude. We introduce Additively Compositional Latent Action Model (AC-LAM), which enforces scene-wise additive composition structure over short horizons on the latent action space. These AC constraints encourage simple algebraic structure in the latent action space~(identity, inverse, cycle consistency) and suppress information that does not compose additively. Empirically, AC-LAM learns more structured, motion-specific, and displacement-calibrated latent actions and provides stronger supervision for downstream policy learning, outperforming state-of-the-art LAMs across simulated and real-world tabletop tasks. |
| title | Learning Additively Compositional Latent Actions for Embodied AI |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2604.03340 |