Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.25263 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909987269246976 |
|---|---|
| author | Miao, Yang Zaech, Jan-Nico Wang, Xi Despinoy, Fabien Paudel, Danda Pani Van Gool, Luc |
| author_facet | Miao, Yang Zaech, Jan-Nico Wang, Xi Despinoy, Fabien Paudel, Danda Pani Van Gool, Luc |
| contents | We propose LangHOPS, the first Multimodal Large Language Model (MLLM) based framework for open-vocabulary object-part instance segmentation. Given an image, LangHOPS can jointly detect and segment hierarchical object and part instances from open-vocabulary candidate categories. Unlike prior approaches that rely on heuristic or learnable visual grouping, our approach grounds object-part hierarchies in language space. It integrates the MLLM into the object-part parsing pipeline to leverage its rich knowledge and reasoning capabilities, and link multi-granularity concepts within the hierarchies. We evaluate LangHOPS across multiple challenging scenarios, including in-domain and cross-dataset object-part instance segmentation, and zero-shot semantic segmentation. LangHOPS achieves state-of-the-art results, surpassing previous methods by 5.5% Average Precision (AP) (in-domain) and 4.8% (cross-dataset) on the PartImageNet dataset and by 2.5% mIOU on unseen object parts in ADE20K (zero-shot). Ablation studies further validate the effectiveness of the language-grounded hierarchy and MLLM driven part query refinement strategy. The code will be released here. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_25263 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation Miao, Yang Zaech, Jan-Nico Wang, Xi Despinoy, Fabien Paudel, Danda Pani Van Gool, Luc Computer Vision and Pattern Recognition We propose LangHOPS, the first Multimodal Large Language Model (MLLM) based framework for open-vocabulary object-part instance segmentation. Given an image, LangHOPS can jointly detect and segment hierarchical object and part instances from open-vocabulary candidate categories. Unlike prior approaches that rely on heuristic or learnable visual grouping, our approach grounds object-part hierarchies in language space. It integrates the MLLM into the object-part parsing pipeline to leverage its rich knowledge and reasoning capabilities, and link multi-granularity concepts within the hierarchies. We evaluate LangHOPS across multiple challenging scenarios, including in-domain and cross-dataset object-part instance segmentation, and zero-shot semantic segmentation. LangHOPS achieves state-of-the-art results, surpassing previous methods by 5.5% Average Precision (AP) (in-domain) and 4.8% (cross-dataset) on the PartImageNet dataset and by 2.5% mIOU on unseen object parts in ADE20K (zero-shot). Ablation studies further validate the effectiveness of the language-grounded hierarchy and MLLM driven part query refinement strategy. The code will be released here. |
| title | LangHOPS: Language Grounded Hierarchical Open-Vocabulary Part Segmentation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.25263 |