GenMask: Adapting DiT for Segmentation via Direct Mask Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908913719312384 |
|---|---|
| author | Yang, Yuhuan Zhuang, Xianwei Cai, Yuxuan Ma, Chaofan Bai, Shuai Yao, Jiangchao Zhang, Ya Lin, Junyang Wang, Yanfeng |
| author_facet | Yang, Yuhuan Zhuang, Xianwei Cai, Yuxuan Ma, Chaofan Bai, Shuai Yao, Jiangchao Zhang, Ya Lin, Junyang Wang, Yanfeng |
| contents | Recent approaches for segmentation have leveraged pretrained generative models as feature extractors, treating segmentation as a downstream adaptation task via indirect feature retrieval. This implicit use suffers from a fundamental misalignment in representation. It also depends heavily on indirect feature extraction pipelines, which complicate the workflow and limit adaptation. In this paper, we argue that instead of indirect adaptation, segmentation tasks should be trained directly in a generative manner. We identify a key obstacle to this unified formulation: VAE latents of binary masks are sharply distributed, noise robust, and linearly separable, distinct from natural image latents. To bridge this gap, we introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation and moderate noise for image generation, enabling harmonious joint training. We present GenMask, a DiT trains to generate black-and-white segmentation masks as well as colorful images in RGB space under the original generative objective. GenMask preserves the original DiT architecture while removing the need of feature extraction pipelines tailored for segmentation tasks. Empirically, GenMask attains state-of-the-art performance on referring and reasoning segmentation benchmarks and ablations quantify the contribution of each component. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23906 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | GenMask: Adapting DiT for Segmentation via Direct Mask Generation Yang, Yuhuan Zhuang, Xianwei Cai, Yuxuan Ma, Chaofan Bai, Shuai Yao, Jiangchao Zhang, Ya Lin, Junyang Wang, Yanfeng Computer Vision and Pattern Recognition Recent approaches for segmentation have leveraged pretrained generative models as feature extractors, treating segmentation as a downstream adaptation task via indirect feature retrieval. This implicit use suffers from a fundamental misalignment in representation. It also depends heavily on indirect feature extraction pipelines, which complicate the workflow and limit adaptation. In this paper, we argue that instead of indirect adaptation, segmentation tasks should be trained directly in a generative manner. We identify a key obstacle to this unified formulation: VAE latents of binary masks are sharply distributed, noise robust, and linearly separable, distinct from natural image latents. To bridge this gap, we introduce timesteps sampling strategy for binary masks that emphasizes extreme noise levels for segmentation and moderate noise for image generation, enabling harmonious joint training. We present GenMask, a DiT trains to generate black-and-white segmentation masks as well as colorful images in RGB space under the original generative objective. GenMask preserves the original DiT architecture while removing the need of feature extraction pipelines tailored for segmentation tasks. Empirically, GenMask attains state-of-the-art performance on referring and reasoning segmentation benchmarks and ablations quantify the contribution of each component. |
| title | GenMask: Adapting DiT for Segmentation via Direct Mask Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2603.23906 |