Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.03588 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909094473891840 |
|---|---|
| author | Shen, Yan Ji, Zhanghexuan Ma, Chunwei Gao, Mingchen |
| author_facet | Shen, Yan Ji, Zhanghexuan Ma, Chunwei Gao, Mingchen |
| contents | Domain adversarial adaptation in a continual setting poses a significant challenge due to the limitations on accessing previous source domain data. Despite extensive research in continual learning, the task of adversarial adaptation cannot be effectively accomplished using only a small number of stored source domain data, which is a standard setting in memory replay approaches. This limitation arises from the erroneous empirical estimation of $\gH$-divergence with few source domain samples. To tackle this problem, we propose a double-head discriminator algorithm, by introducing an addition source-only domain discriminator that are trained solely on source learning phase. We prove that with the introduction of a pre-trained source-only domain discriminator, the empirical estimation error of $\gH$-divergence related adversarial loss is reduced from the source domain side. Further experiments on existing domain adaptation benchmark show that our proposed algorithm achieves more than 2$\%$ improvement on all categories of target domain adaptation task while significantly mitigating the forgetting on source domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_03588 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Continual Domain Adversarial Adaptation via Double-Head Discriminators Shen, Yan Ji, Zhanghexuan Ma, Chunwei Gao, Mingchen Machine Learning Artificial Intelligence Domain adversarial adaptation in a continual setting poses a significant challenge due to the limitations on accessing previous source domain data. Despite extensive research in continual learning, the task of adversarial adaptation cannot be effectively accomplished using only a small number of stored source domain data, which is a standard setting in memory replay approaches. This limitation arises from the erroneous empirical estimation of $\gH$-divergence with few source domain samples. To tackle this problem, we propose a double-head discriminator algorithm, by introducing an addition source-only domain discriminator that are trained solely on source learning phase. We prove that with the introduction of a pre-trained source-only domain discriminator, the empirical estimation error of $\gH$-divergence related adversarial loss is reduced from the source domain side. Further experiments on existing domain adaptation benchmark show that our proposed algorithm achieves more than 2$\%$ improvement on all categories of target domain adaptation task while significantly mitigating the forgetting on source domain. |
| title | Continual Domain Adversarial Adaptation via Double-Head Discriminators |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.03588 |