Towards Agnostic and Holistic Universal Image Segmentation with Bit Diffusion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917186788917248 |
|---|---|
| author | Christensen, Jakob Lønborg Hannemose, Morten Rieger Dahl, Anders Bjorholm Dahl, Vedrana Andersen |
| author_facet | Christensen, Jakob Lønborg Hannemose, Morten Rieger Dahl, Anders Bjorholm Dahl, Vedrana Andersen |
| contents | This paper introduces a diffusion-based framework for universal image segmentation, making agnostic segmentation possible without depending on mask-based frameworks and instead predicting the full segmentation in a holistic manner. We present several key adaptations to diffusion models, which are important in this discrete setting. Notably, we show that a location-aware palette with our 2D gray code ordering improves performance. Adding a final tanh activation function is crucial for discrete data. On optimizing diffusion parameters, the sigmoid loss weighting consistently outperforms alternatives, regardless of the prediction type used, and we settle on x-prediction. While our current model does not yet surpass leading mask-based architectures, it narrows the performance gap and introduces unique capabilities, such as principled ambiguity modeling, that these models lack. All models were trained from scratch, and we believe that combining our proposed improvements with large-scale pretraining or promptable conditioning could lead to competitive models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_02881 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Towards Agnostic and Holistic Universal Image Segmentation with Bit Diffusion Christensen, Jakob Lønborg Hannemose, Morten Rieger Dahl, Anders Bjorholm Dahl, Vedrana Andersen Computer Vision and Pattern Recognition This paper introduces a diffusion-based framework for universal image segmentation, making agnostic segmentation possible without depending on mask-based frameworks and instead predicting the full segmentation in a holistic manner. We present several key adaptations to diffusion models, which are important in this discrete setting. Notably, we show that a location-aware palette with our 2D gray code ordering improves performance. Adding a final tanh activation function is crucial for discrete data. On optimizing diffusion parameters, the sigmoid loss weighting consistently outperforms alternatives, regardless of the prediction type used, and we settle on x-prediction. While our current model does not yet surpass leading mask-based architectures, it narrows the performance gap and introduces unique capabilities, such as principled ambiguity modeling, that these models lack. All models were trained from scratch, and we believe that combining our proposed improvements with large-scale pretraining or promptable conditioning could lead to competitive models. |
| title | Towards Agnostic and Holistic Universal Image Segmentation with Bit Diffusion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2601.02881 |