OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918465902739456 |
|---|---|
| author | Liang, Zhuding Yan, Tianyi Chen, Dubing Zheng, Jiasen Zheng, Huan Xu, Cheng-zhong Wang, Yida Zhan, Kun Shen, Jianbing |
| author_facet | Liang, Zhuding Yan, Tianyi Chen, Dubing Zheng, Jiasen Zheng, Huan Xu, Cheng-zhong Wang, Yida Zhan, Kun Shen, Jianbing |
| contents | Generative world models increasingly rely on 4D occupancy for realistic autonomous driving simulation. However, existing generation frameworks depend on rigid geometric conditions (e.g., explicit trajectories) or simplistic attribute-level text, failing to orchestrate complex, sequential multi-agent interactions. To address this semantic-spatiotemporal gap, we propose OccDirector, a pioneering framework that generates 4D occupancy dynamics conditioned solely on natural language. Operating as a ``scenario director'', OccDirector maps natural language scripts into physically plausible voxel dynamics without requiring geometric priors. Technically, it employs a VLM-driven Spatio-Temporal MMDiT equipped with a history-prefix anchoring strategy to ensure long-horizon interaction consistency. Furthermore, we introduce OccInteract-85k, a novel dataset uniquely annotated with multi-level language instructions: ranging from static layouts to intricate multi-agent behaviors, alongside a novel VLM-based evaluation benchmark. Extensive experiments demonstrate that OccDirector achieves state-of-the-art generation quality and unprecedented instruction-following capabilities, successfully shifting the paradigm from appearance synthesis to language-driven behavior orchestration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22240 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space Liang, Zhuding Yan, Tianyi Chen, Dubing Zheng, Jiasen Zheng, Huan Xu, Cheng-zhong Wang, Yida Zhan, Kun Shen, Jianbing Computer Vision and Pattern Recognition Generative world models increasingly rely on 4D occupancy for realistic autonomous driving simulation. However, existing generation frameworks depend on rigid geometric conditions (e.g., explicit trajectories) or simplistic attribute-level text, failing to orchestrate complex, sequential multi-agent interactions. To address this semantic-spatiotemporal gap, we propose OccDirector, a pioneering framework that generates 4D occupancy dynamics conditioned solely on natural language. Operating as a ``scenario director'', OccDirector maps natural language scripts into physically plausible voxel dynamics without requiring geometric priors. Technically, it employs a VLM-driven Spatio-Temporal MMDiT equipped with a history-prefix anchoring strategy to ensure long-horizon interaction consistency. Furthermore, we introduce OccInteract-85k, a novel dataset uniquely annotated with multi-level language instructions: ranging from static layouts to intricate multi-agent behaviors, alongside a novel VLM-based evaluation benchmark. Extensive experiments demonstrate that OccDirector achieves state-of-the-art generation quality and unprecedented instruction-following capabilities, successfully shifting the paradigm from appearance synthesis to language-driven behavior orchestration. |
| title | OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.22240 |