Invariant Representation via Decoupling Style and Spurious Features from Images
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914736971448320 |
|---|---|
| author | Li, Ruimeng Pu, Yuanhao Li, Zhaoyi Xie, Hong Lian, Defu |
| author_facet | Li, Ruimeng Pu, Yuanhao Li, Zhaoyi Xie, Hong Lian, Defu |
| contents | This paper considers the out-of-distribution (OOD) generalization problem under the setting that both style distribution shift and spurious features exist and domain labels are missing. This setting frequently arises in real-world applications and is underlooked because previous approaches mainly handle either of these two factors. The critical challenge is decoupling style and spurious features in the absence of domain labels. To address this challenge, we first propose a structural causal model (SCM) for the image generation process, which captures both style distribution shift and spurious features. The proposed SCM enables us to design a new framework called IRSS, which can gradually separate style distribution and spurious features from images by introducing adversarial neural networks and multi-environment optimization, thus achieving OOD generalization. Moreover, it does not require additional supervision (e.g., domain labels) other than the images and their corresponding labels. Experiments on benchmark datasets demonstrate that IRSS outperforms traditional OOD methods and solves the problem of Invariant risk minimization (IRM) degradation, enabling the extraction of invariant features under distribution shift. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2312_06226 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Invariant Representation via Decoupling Style and Spurious Features from Images Li, Ruimeng Pu, Yuanhao Li, Zhaoyi Xie, Hong Lian, Defu Computer Vision and Pattern Recognition Artificial Intelligence I.2.6; I.2.10 This paper considers the out-of-distribution (OOD) generalization problem under the setting that both style distribution shift and spurious features exist and domain labels are missing. This setting frequently arises in real-world applications and is underlooked because previous approaches mainly handle either of these two factors. The critical challenge is decoupling style and spurious features in the absence of domain labels. To address this challenge, we first propose a structural causal model (SCM) for the image generation process, which captures both style distribution shift and spurious features. The proposed SCM enables us to design a new framework called IRSS, which can gradually separate style distribution and spurious features from images by introducing adversarial neural networks and multi-environment optimization, thus achieving OOD generalization. Moreover, it does not require additional supervision (e.g., domain labels) other than the images and their corresponding labels. Experiments on benchmark datasets demonstrate that IRSS outperforms traditional OOD methods and solves the problem of Invariant risk minimization (IRM) degradation, enabling the extraction of invariant features under distribution shift. |
| title | Invariant Representation via Decoupling Style and Spurious Features from Images |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence I.2.6; I.2.10 |
| url | https://arxiv.org/abs/2312.06226 |