OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916203641962496 |
|---|---|
| author | Shrestha, Robik Kafle, Kushal Kanan, Christopher |
| author_facet | Shrestha, Robik Kafle, Kushal Kanan, Christopher |
| contents | Dataset bias and spurious correlations can significantly impair generalization in deep neural networks. Many prior efforts have addressed this problem using either alternative loss functions or sampling strategies that focus on rare patterns. We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Specifically, we propose OccamNets, which are biased to favor simpler solutions by design. OccamNets have two inductive biases. First, they are biased to use as little network depth as needed for an individual example. Second, they are biased toward using fewer image locations for prediction. While OccamNets are biased toward simpler hypotheses, they can learn more complex hypotheses if necessary. In experiments, OccamNets outperform or rival state-of-the-art methods run on architectures that do not incorporate these inductive biases. Furthermore, we demonstrate that when the state-of-the-art debiasing methods are combined with OccamNets results further improve. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2204_02426 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses Shrestha, Robik Kafle, Kushal Kanan, Christopher Machine Learning Dataset bias and spurious correlations can significantly impair generalization in deep neural networks. Many prior efforts have addressed this problem using either alternative loss functions or sampling strategies that focus on rare patterns. We propose a new direction: modifying the network architecture to impose inductive biases that make the network robust to dataset bias. Specifically, we propose OccamNets, which are biased to favor simpler solutions by design. OccamNets have two inductive biases. First, they are biased to use as little network depth as needed for an individual example. Second, they are biased toward using fewer image locations for prediction. While OccamNets are biased toward simpler hypotheses, they can learn more complex hypotheses if necessary. In experiments, OccamNets outperform or rival state-of-the-art methods run on architectures that do not incorporate these inductive biases. Furthermore, we demonstrate that when the state-of-the-art debiasing methods are combined with OccamNets results further improve. |
| title | OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2204.02426 |