3DCity-LLM: Empowering Multi-modality Large Language Models for 3D City-scale Perception and Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918407058751488 |
|---|---|
| author | Chen, Yiping Li, Jinpeng Ke, Wenyu Luo, Yang Ouyang, Jie He, Zhongjie Liu, Li Fan, Hongchao Wu, Hao |
| author_facet | Chen, Yiping Li, Jinpeng Ke, Wenyu Luo, Yang Ouyang, Jie He, Zhongjie Liu, Li Fan, Hongchao Wu, Hao |
| contents | While multi-modality large language models excel in object-centric or indoor scenarios, scaling them to 3D city-scale environments remains a formidable challenge. To bridge this gap, we propose 3DCity-LLM, a unified framework designed for 3D city-scale vision-language perception and understanding. 3DCity-LLM employs a coarse-to-fine feature encoding strategy comprising three parallel branches for target object, inter-object relationship, and global scene. To facilitate large-scale training, we introduce 3DCity-LLM-1.2M dataset that comprises approximately 1.2 million high-quality samples across seven representative task categories, ranging from fine-grained object analysis to multi-faceted scene planning. This strictly quality-controlled dataset integrates explicit 3D numerical information and diverse user-oriented simulations, enriching the question-answering diversity and realism of urban scenarios. Furthermore, we apply a multi-dimensional protocol based on text-similarity metrics and LLM-based semantic assessment to ensure faithful and comprehensive evaluations for all methods. Extensive experiments on two benchmarks demonstrate that 3DCity-LLM significantly outperforms existing state-of-the-art methods, offering a promising and meaningful direction for advancing spatial reasoning and urban intelligence. The source code and dataset are available at https://github.com/SYSU-3DSTAILab/3D-City-LLM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23447 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | 3DCity-LLM: Empowering Multi-modality Large Language Models for 3D City-scale Perception and Understanding Chen, Yiping Li, Jinpeng Ke, Wenyu Luo, Yang Ouyang, Jie He, Zhongjie Liu, Li Fan, Hongchao Wu, Hao Computer Vision and Pattern Recognition Artificial Intelligence I.2.10 While multi-modality large language models excel in object-centric or indoor scenarios, scaling them to 3D city-scale environments remains a formidable challenge. To bridge this gap, we propose 3DCity-LLM, a unified framework designed for 3D city-scale vision-language perception and understanding. 3DCity-LLM employs a coarse-to-fine feature encoding strategy comprising three parallel branches for target object, inter-object relationship, and global scene. To facilitate large-scale training, we introduce 3DCity-LLM-1.2M dataset that comprises approximately 1.2 million high-quality samples across seven representative task categories, ranging from fine-grained object analysis to multi-faceted scene planning. This strictly quality-controlled dataset integrates explicit 3D numerical information and diverse user-oriented simulations, enriching the question-answering diversity and realism of urban scenarios. Furthermore, we apply a multi-dimensional protocol based on text-similarity metrics and LLM-based semantic assessment to ensure faithful and comprehensive evaluations for all methods. Extensive experiments on two benchmarks demonstrate that 3DCity-LLM significantly outperforms existing state-of-the-art methods, offering a promising and meaningful direction for advancing spatial reasoning and urban intelligence. The source code and dataset are available at https://github.com/SYSU-3DSTAILab/3D-City-LLM. |
| title | 3DCity-LLM: Empowering Multi-modality Large Language Models for 3D City-scale Perception and Understanding |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence I.2.10 |
| url | https://arxiv.org/abs/2603.23447 |