Semantic Self-adaptation: Enhancing Generalization with a Single Sample
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912009162850304 |
|---|---|
| author | Bahmani, Sherwin Hahn, Oliver Zamfir, Eduard Araslanov, Nikita Cremers, Daniel Roth, Stefan |
| author_facet | Bahmani, Sherwin Hahn, Oliver Zamfir, Eduard Araslanov, Nikita Cremers, Daniel Roth, Stefan |
| contents | The lack of out-of-domain generalization is a critical weakness of deep networks for semantic segmentation. Previous studies relied on the assumption of a static model, i. e., once the training process is complete, model parameters remain fixed at test time. In this work, we challenge this premise with a self-adaptive approach for semantic segmentation that adjusts the inference process to each input sample. Self-adaptation operates on two levels. First, it fine-tunes the parameters of convolutional layers to the input image using consistency regularization. Second, in Batch Normalization layers, self-adaptation interpolates between the training and the reference distribution derived from a single test sample. Despite both techniques being well known in the literature, their combination sets new state-of-the-art accuracy on synthetic-to-real generalization benchmarks. Our empirical study suggests that self-adaptation may complement the established practice of model regularization at training time for improving deep network generalization to out-of-domain data. Our code and pre-trained models are available at https://github.com/visinf/self-adaptive. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_05788 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Semantic Self-adaptation: Enhancing Generalization with a Single Sample Bahmani, Sherwin Hahn, Oliver Zamfir, Eduard Araslanov, Nikita Cremers, Daniel Roth, Stefan Computer Vision and Pattern Recognition The lack of out-of-domain generalization is a critical weakness of deep networks for semantic segmentation. Previous studies relied on the assumption of a static model, i. e., once the training process is complete, model parameters remain fixed at test time. In this work, we challenge this premise with a self-adaptive approach for semantic segmentation that adjusts the inference process to each input sample. Self-adaptation operates on two levels. First, it fine-tunes the parameters of convolutional layers to the input image using consistency regularization. Second, in Batch Normalization layers, self-adaptation interpolates between the training and the reference distribution derived from a single test sample. Despite both techniques being well known in the literature, their combination sets new state-of-the-art accuracy on synthetic-to-real generalization benchmarks. Our empirical study suggests that self-adaptation may complement the established practice of model regularization at training time for improving deep network generalization to out-of-domain data. Our code and pre-trained models are available at https://github.com/visinf/self-adaptive. |
| title | Semantic Self-adaptation: Enhancing Generalization with a Single Sample |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2208.05788 |