Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.11682 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910335726780416 |
|---|---|
| author | Chaudhuri, Abhra Georgescu, Serban Dutta, Anjan |
| author_facet | Chaudhuri, Abhra Georgescu, Serban Dutta, Anjan |
| contents | Invariance learning algorithms that conditionally filter out domain-specific random variables as distractors, do so based only on the data semantics, and not the target domain under evaluation. We show that a provably optimal and sample-efficient way of learning conditional invariances is by relaxing the invariance criterion to be non-commutatively directed towards the target domain. Under domain asymmetry, i.e., when the target domain contains semantically relevant information absent in the source, the risk of the encoder $φ^*$ that is optimal on average across domains is strictly lower-bounded by the risk of the target-specific optimal encoder $Φ^*_τ$. We prove that non-commutativity steers the optimization towards $Φ^*_τ$ instead of $φ^*$, bringing the $\mathcal{H}$-divergence between domains down to zero, leading to a stricter bound on the target risk. Both our theory and experiments demonstrate that non-commutative invariance (NCI) can leverage source domain samples to meet the sample complexity needs of learning $Φ^*_τ$, surpassing SOTA invariance learning algorithms for domain adaptation, at times by over $2\%$, approaching the performance of an oracle. Implementation is available at https://github.com/abhrac/nci. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11682 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning Conditional Invariances through Non-Commutativity Chaudhuri, Abhra Georgescu, Serban Dutta, Anjan Machine Learning Computer Vision and Pattern Recognition Invariance learning algorithms that conditionally filter out domain-specific random variables as distractors, do so based only on the data semantics, and not the target domain under evaluation. We show that a provably optimal and sample-efficient way of learning conditional invariances is by relaxing the invariance criterion to be non-commutatively directed towards the target domain. Under domain asymmetry, i.e., when the target domain contains semantically relevant information absent in the source, the risk of the encoder $φ^*$ that is optimal on average across domains is strictly lower-bounded by the risk of the target-specific optimal encoder $Φ^*_τ$. We prove that non-commutativity steers the optimization towards $Φ^*_τ$ instead of $φ^*$, bringing the $\mathcal{H}$-divergence between domains down to zero, leading to a stricter bound on the target risk. Both our theory and experiments demonstrate that non-commutative invariance (NCI) can leverage source domain samples to meet the sample complexity needs of learning $Φ^*_τ$, surpassing SOTA invariance learning algorithms for domain adaptation, at times by over $2\%$, approaching the performance of an oracle. Implementation is available at https://github.com/abhrac/nci. |
| title | Learning Conditional Invariances through Non-Commutativity |
| topic | Machine Learning Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.11682 |