Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.13484 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915999446466560 |
|---|---|
| author | Liu, Liu Kudaeva, Alexandra Cipriano, Marco Ghannam, Fatimeh Al Tan, Freya de Melo, Gerard Sevtsuk, Andres |
| author_facet | Liu, Liu Kudaeva, Alexandra Cipriano, Marco Ghannam, Fatimeh Al Tan, Freya de Melo, Gerard Sevtsuk, Andres |
| contents | Understanding group-level social interactions in public spaces is crucial for urban planning, informing the design of socially vibrant and inclusive environments. Detecting such interactions from images involves interpreting subtle visual cues such as relations, proximity, and co-movement - semantically complex signals that go beyond traditional object detection. To address this challenge, we introduce a social group region detection task, which requires inferring and spatially grounding visual regions defined by abstract interpersonal relations. We propose MINGLE (Modeling INterpersonal Group-Level Engagement), a modular three-stage pipeline that integrates: (1) off-the-shelf human detection and depth estimation, (2) VLM-based reasoning to classify pairwise social affiliation, and (3) a lightweight spatial aggregation algorithm to localize socially connected groups. To support this task and encourage future research, we present a new dataset of 100K urban street-view images annotated with bounding boxes and labels for both individuals and socially interacting groups. The annotations combine human-created labels and outputs from the MINGLE pipeline, ensuring semantic richness and broad coverage of real-world scenarios. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13484 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MINGLE: VLMs for Semantically Complex Region Detection in Urban Scenes Liu, Liu Kudaeva, Alexandra Cipriano, Marco Ghannam, Fatimeh Al Tan, Freya de Melo, Gerard Sevtsuk, Andres Computer Vision and Pattern Recognition Computers and Society Understanding group-level social interactions in public spaces is crucial for urban planning, informing the design of socially vibrant and inclusive environments. Detecting such interactions from images involves interpreting subtle visual cues such as relations, proximity, and co-movement - semantically complex signals that go beyond traditional object detection. To address this challenge, we introduce a social group region detection task, which requires inferring and spatially grounding visual regions defined by abstract interpersonal relations. We propose MINGLE (Modeling INterpersonal Group-Level Engagement), a modular three-stage pipeline that integrates: (1) off-the-shelf human detection and depth estimation, (2) VLM-based reasoning to classify pairwise social affiliation, and (3) a lightweight spatial aggregation algorithm to localize socially connected groups. To support this task and encourage future research, we present a new dataset of 100K urban street-view images annotated with bounding boxes and labels for both individuals and socially interacting groups. The annotations combine human-created labels and outputs from the MINGLE pipeline, ensuring semantic richness and broad coverage of real-world scenarios. |
| title | MINGLE: VLMs for Semantically Complex Region Detection in Urban Scenes |
| topic | Computer Vision and Pattern Recognition Computers and Society |
| url | https://arxiv.org/abs/2509.13484 |