Consistency Regularization for Domain Generalization with Logit Attribution Matching
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929382321291264 |
|---|---|
| author | Gao, Han Li, Kaican Xie, Weiyan Lin, Zhi Huang, Yongxiang Wang, Luning Cao, Caleb Chen Zhang, Nevin L. |
| author_facet | Gao, Han Li, Kaican Xie, Weiyan Lin, Zhi Huang, Yongxiang Wang, Luning Cao, Caleb Chen Zhang, Nevin L. |
| contents | Domain generalization (DG) is about training models that generalize well under domain shift. Previous research on DG has been conducted mostly in single-source or multi-source settings. In this paper, we consider a third, lesser-known setting where a training domain is endowed with a collection of pairs of examples that share the same semantic information. Such semantic sharing (SS) pairs can be created via data augmentation and then utilized for consistency regularization (CR). We present a theory showing CR is conducive to DG and propose a novel CR method called Logit Attribution Matching (LAM). We conduct experiments on five DG benchmarks and four pretrained models with SS pairs created by both generic and targeted data augmentation methods. LAM outperforms representative single/multi-source DG methods and various CR methods that leverage SS pairs. The code and data of this project are available at https://github.com/Gaohan123/LAM |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_07888 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Consistency Regularization for Domain Generalization with Logit Attribution Matching Gao, Han Li, Kaican Xie, Weiyan Lin, Zhi Huang, Yongxiang Wang, Luning Cao, Caleb Chen Zhang, Nevin L. Machine Learning Domain generalization (DG) is about training models that generalize well under domain shift. Previous research on DG has been conducted mostly in single-source or multi-source settings. In this paper, we consider a third, lesser-known setting where a training domain is endowed with a collection of pairs of examples that share the same semantic information. Such semantic sharing (SS) pairs can be created via data augmentation and then utilized for consistency regularization (CR). We present a theory showing CR is conducive to DG and propose a novel CR method called Logit Attribution Matching (LAM). We conduct experiments on five DG benchmarks and four pretrained models with SS pairs created by both generic and targeted data augmentation methods. LAM outperforms representative single/multi-source DG methods and various CR methods that leverage SS pairs. The code and data of this project are available at https://github.com/Gaohan123/LAM |
| title | Consistency Regularization for Domain Generalization with Logit Attribution Matching |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.07888 |