HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929632048054272 |
|---|---|
| author | Wang, Zirui Zhao, Xinran Stepputtis, Simon Kim, Woojun Wu, Tongshuang Sycara, Katia Xie, Yaqi |
| author_facet | Wang, Zirui Zhao, Xinran Stepputtis, Simon Kim, Woojun Wu, Tongshuang Sycara, Katia Xie, Yaqi |
| contents | Understanding and predicting human actions has been a long-standing challenge and is a crucial measure of perception in robotics AI. While significant progress has been made in anticipating the future actions of individual agents, prior work has largely overlooked a key aspect of real-world human activity -- interactions. To address this gap in human-like forecasting within multi-agent environments, we present the Hierarchical Memory-Aware Transformer (HiMemFormer), a transformer-based model for online multi-agent action anticipation. HiMemFormer integrates and distributes global memory that captures joint historical information across all agents through a transformer framework, with a hierarchical local memory decoder that interprets agent-specific features based on these global representations using a coarse-to-fine strategy. In contrast to previous approaches, HiMemFormer uniquely hierarchically applies the global context with agent-specific preferences to avoid noisy or redundant information in multi-agent action anticipation. Extensive experiments on various multi-agent scenarios demonstrate the significant performance of HiMemFormer, compared with other state-of-the-art methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_01455 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation Wang, Zirui Zhao, Xinran Stepputtis, Simon Kim, Woojun Wu, Tongshuang Sycara, Katia Xie, Yaqi Computer Vision and Pattern Recognition Multiagent Systems Understanding and predicting human actions has been a long-standing challenge and is a crucial measure of perception in robotics AI. While significant progress has been made in anticipating the future actions of individual agents, prior work has largely overlooked a key aspect of real-world human activity -- interactions. To address this gap in human-like forecasting within multi-agent environments, we present the Hierarchical Memory-Aware Transformer (HiMemFormer), a transformer-based model for online multi-agent action anticipation. HiMemFormer integrates and distributes global memory that captures joint historical information across all agents through a transformer framework, with a hierarchical local memory decoder that interprets agent-specific features based on these global representations using a coarse-to-fine strategy. In contrast to previous approaches, HiMemFormer uniquely hierarchically applies the global context with agent-specific preferences to avoid noisy or redundant information in multi-agent action anticipation. Extensive experiments on various multi-agent scenarios demonstrate the significant performance of HiMemFormer, compared with other state-of-the-art methods. |
| title | HiMemFormer: Hierarchical Memory-Aware Transformer for Multi-Agent Action Anticipation |
| topic | Computer Vision and Pattern Recognition Multiagent Systems |
| url | https://arxiv.org/abs/2411.01455 |