Saved in:
Bibliographic Details
Main Authors: Sun, Jiacheng, Lin, Jiaqi, Hu, Wenlong, Li, Haoyang, Zhou, Xinghong, Mao, Chenghai, Peng, Yan, Li, Xiaomao
Format: Preprint
Published: 2026
Subjects:
Online Access:https://arxiv.org/abs/2603.09245
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866908875960090624
author Sun, Jiacheng
Lin, Jiaqi
Hu, Wenlong
Li, Haoyang
Zhou, Xinghong
Mao, Chenghai
Peng, Yan
Li, Xiaomao
author_facet Sun, Jiacheng
Lin, Jiaqi
Hu, Wenlong
Li, Haoyang
Zhou, Xinghong
Mao, Chenghai
Peng, Yan
Li, Xiaomao
contents One of the bottlenecks for instance segmentation today lies in the conflicting requirements of high-resolution inputs and lightweight, real-time inference. To address this bottleneck, we present a Polygon Detection Transformer (Poly-DETR) to reformulate instance segmentation as sparse vertex regression via Polar Representation, thereby eliminating the reliance on dense pixel-wise mask prediction. Considering the box-to-polygon reference shift in Detection Transformers, we propose Polar Deformable Attention and Position-Aware Training Scheme to dynamically update supervision and focus attention on boundary cues. Compared with state-of-the-art polar-based methods, Poly-DETR achieves a 4.7 mAP improvement on MS COCO test-dev. Moreover, we construct a parallel mask-based counterpart to support a systematic comparison between polar and mask representations. Experimental results show that Poly-DETR is more lightweight in high-resolution scenarios, reducing memory consumption by almost half on Cityscapes dataset. Notably, on PanNuke (cell segmentation) and SpaceNet (building footprints) datasets, Poly-DETR surpasses its mask-based counterpart on all metrics, which validates its advantage on regular-shaped instances in domain-specific settings.
format Preprint
id arxiv_https___arxiv_org_abs_2603_09245
institution arXiv
publishDate 2026
record_format arxiv
spellingShingle Towards Instance Segmentation with Polygon Detection Transformers
Sun, Jiacheng
Lin, Jiaqi
Hu, Wenlong
Li, Haoyang
Zhou, Xinghong
Mao, Chenghai
Peng, Yan
Li, Xiaomao
Computer Vision and Pattern Recognition
One of the bottlenecks for instance segmentation today lies in the conflicting requirements of high-resolution inputs and lightweight, real-time inference. To address this bottleneck, we present a Polygon Detection Transformer (Poly-DETR) to reformulate instance segmentation as sparse vertex regression via Polar Representation, thereby eliminating the reliance on dense pixel-wise mask prediction. Considering the box-to-polygon reference shift in Detection Transformers, we propose Polar Deformable Attention and Position-Aware Training Scheme to dynamically update supervision and focus attention on boundary cues. Compared with state-of-the-art polar-based methods, Poly-DETR achieves a 4.7 mAP improvement on MS COCO test-dev. Moreover, we construct a parallel mask-based counterpart to support a systematic comparison between polar and mask representations. Experimental results show that Poly-DETR is more lightweight in high-resolution scenarios, reducing memory consumption by almost half on Cityscapes dataset. Notably, on PanNuke (cell segmentation) and SpaceNet (building footprints) datasets, Poly-DETR surpasses its mask-based counterpart on all metrics, which validates its advantage on regular-shaped instances in domain-specific settings.
title Towards Instance Segmentation with Polygon Detection Transformers
topic Computer Vision and Pattern Recognition
url https://arxiv.org/abs/2603.09245