Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.12734 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918360516657152 |
|---|---|
| author | Wang, Junbo Tan, Haofeng Liao, Bowen Jiang, Albert Fei, Teng Huang, Qixing Zhou, Bing Tu, Zhengzhong Ye, Shan Kang, Yuhao |
| author_facet | Wang, Junbo Tan, Haofeng Liao, Bowen Jiang, Albert Fei, Teng Huang, Qixing Zhou, Bing Tu, Zhengzhong Ye, Shan Kang, Yuhao |
| contents | Recent audio-to-image models have shown impressive performance in generating images of specific objects conditioned on their corresponding sounds. However, these models fail to reconstruct real-world landscapes conditioned on environmental soundscapes. To address this gap, we present Geo-contextual Soundscape-to-Landscape (GeoS2L) generation, a novel and practically significant task that aims to synthesize geographically realistic landscape images from environmental soundscapes. To support this task, we construct two large-scale geo-contextual multi-modal datasets, SoundingSVI and SonicUrban, which pair diverse environmental soundscapes with real-world landscape images. We propose SounDiT, a diffusion transformer (DiT)-based model that incorporates environmental soundscapes and geo-contextual scene conditioning to synthesize geographically coherent landscape images. Furthermore, we propose the Place Similarity Score (PSS), a practically-informed geo-contextual evaluation framework to measure consistency between input soundscapes and generated landscape images. Extensive experiments demonstrate that SounDiT outperforms existing baselines in the GeoS2L, while the PSS effectively captures multi-level generation consistency across element, scene,and human perception. Project page: https://gisense.github.io/SounDiT-Page/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_12734 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SounDiT: Geo-Contextual Soundscape-to-Landscape Generation Wang, Junbo Tan, Haofeng Liao, Bowen Jiang, Albert Fei, Teng Huang, Qixing Zhou, Bing Tu, Zhengzhong Ye, Shan Kang, Yuhao Sound Artificial Intelligence Graphics Human-Computer Interaction Audio and Speech Processing Recent audio-to-image models have shown impressive performance in generating images of specific objects conditioned on their corresponding sounds. However, these models fail to reconstruct real-world landscapes conditioned on environmental soundscapes. To address this gap, we present Geo-contextual Soundscape-to-Landscape (GeoS2L) generation, a novel and practically significant task that aims to synthesize geographically realistic landscape images from environmental soundscapes. To support this task, we construct two large-scale geo-contextual multi-modal datasets, SoundingSVI and SonicUrban, which pair diverse environmental soundscapes with real-world landscape images. We propose SounDiT, a diffusion transformer (DiT)-based model that incorporates environmental soundscapes and geo-contextual scene conditioning to synthesize geographically coherent landscape images. Furthermore, we propose the Place Similarity Score (PSS), a practically-informed geo-contextual evaluation framework to measure consistency between input soundscapes and generated landscape images. Extensive experiments demonstrate that SounDiT outperforms existing baselines in the GeoS2L, while the PSS effectively captures multi-level generation consistency across element, scene,and human perception. Project page: https://gisense.github.io/SounDiT-Page/ |
| title | SounDiT: Geo-Contextual Soundscape-to-Landscape Generation |
| topic | Sound Artificial Intelligence Graphics Human-Computer Interaction Audio and Speech Processing |
| url | https://arxiv.org/abs/2505.12734 |