Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.19228 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914283490639872 |
|---|---|
| author | Song, Tianhui Lu, Haoyu Yang, Hao Sui, Lin Wu, Haoning Zhou, Zaida Huang, Zhiqi Bao, Yiping Charles, Y. Zhou, Xinyu Wang, Limin |
| author_facet | Song, Tianhui Lu, Haoyu Yang, Hao Sui, Lin Wu, Haoning Zhou, Zaida Huang, Zhiqi Bao, Yiping Charles, Y. Zhou, Xinyu Wang, Limin |
| contents | We present SimpleSeg, a strikingly simple yet highly effective approach to endow Multimodal Large Language Models (MLLMs) with native pixel-level perception. Our method reframes segmentation as a simple sequence generation problem: the model directly predicts sequences of points (textual coordinates) delineating object boundaries, entirely within its language space. To achieve high fidelity, we introduce a two-stage SF$\to$RL training pipeline, where Reinforcement Learning with an IoU-based reward refines the point sequences to accurately match ground-truth contours. We find that the standard MLLM architecture possesses a strong, inherent capacity for low-level perception that can be unlocked without any specialized architecture. On segmentation benchmarks, SimpleSeg achieves performance that is comparable to, and often surpasses, methods relying on complex, task-specific designs. This work lays out that precise spatial understanding can emerge from simple point prediction, challenging the prevailing need for auxiliary components and paving the way for more unified and capable VLMs. Homepage: https://simpleseg.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19228 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Pixel-Level VLM Perception via Simple Points Prediction Song, Tianhui Lu, Haoyu Yang, Hao Sui, Lin Wu, Haoning Zhou, Zaida Huang, Zhiqi Bao, Yiping Charles, Y. Zhou, Xinyu Wang, Limin Computer Vision and Pattern Recognition We present SimpleSeg, a strikingly simple yet highly effective approach to endow Multimodal Large Language Models (MLLMs) with native pixel-level perception. Our method reframes segmentation as a simple sequence generation problem: the model directly predicts sequences of points (textual coordinates) delineating object boundaries, entirely within its language space. To achieve high fidelity, we introduce a two-stage SF$\to$RL training pipeline, where Reinforcement Learning with an IoU-based reward refines the point sequences to accurately match ground-truth contours. We find that the standard MLLM architecture possesses a strong, inherent capacity for low-level perception that can be unlocked without any specialized architecture. On segmentation benchmarks, SimpleSeg achieves performance that is comparable to, and often surpasses, methods relying on complex, task-specific designs. This work lays out that precise spatial understanding can emerge from simple point prediction, challenging the prevailing need for auxiliary components and paving the way for more unified and capable VLMs. Homepage: https://simpleseg.github.io/ |
| title | Towards Pixel-Level VLM Perception via Simple Points Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.19228 |