ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911909546033152 |
|---|---|
| author | Ghosh, Sreyan Evuru, Chandra Kiran Reddy Kumar, Sonal Tyagi, Utkarsh Singh, Sakshi Chowdhury, Sanjoy Manocha, Dinesh |
| author_facet | Ghosh, Sreyan Evuru, Chandra Kiran Reddy Kumar, Sonal Tyagi, Utkarsh Singh, Sakshi Chowdhury, Sanjoy Manocha, Dinesh |
| contents | Neural image classifiers can often learn to make predictions by overly relying on non-predictive features that are spuriously correlated with the class labels in the training data. This leads to poor performance in real-world atypical scenarios where such features are absent. This paper presents ASPIRE (Language-guided Data Augmentation for SPurIous correlation REmoval), a simple yet effective solution for supplementing the training dataset with images without spurious features, for robust learning against spurious correlations via better generalization. ASPIRE, guided by language at various steps, can generate non-spurious images without requiring any group labeling or existing non-spurious images in the training set. Precisely, we employ LLMs to first extract foreground and background features from textual descriptions of an image, followed by advanced language-guided image editing to discover the features that are spuriously correlated with the class label. Finally, we personalize a text-to-image generation model using the edited images to generate diverse in-domain images without spurious features. ASPIRE is complementary to all prior robust training methods in literature, and we demonstrate its effectiveness across 4 datasets and 9 baselines and show that ASPIRE improves the worst-group classification accuracy of prior methods by 1% - 38%. We also contribute a novel test set for the challenging Hard ImageNet dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_10103 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlations Ghosh, Sreyan Evuru, Chandra Kiran Reddy Kumar, Sonal Tyagi, Utkarsh Singh, Sakshi Chowdhury, Sanjoy Manocha, Dinesh Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Neural image classifiers can often learn to make predictions by overly relying on non-predictive features that are spuriously correlated with the class labels in the training data. This leads to poor performance in real-world atypical scenarios where such features are absent. This paper presents ASPIRE (Language-guided Data Augmentation for SPurIous correlation REmoval), a simple yet effective solution for supplementing the training dataset with images without spurious features, for robust learning against spurious correlations via better generalization. ASPIRE, guided by language at various steps, can generate non-spurious images without requiring any group labeling or existing non-spurious images in the training set. Precisely, we employ LLMs to first extract foreground and background features from textual descriptions of an image, followed by advanced language-guided image editing to discover the features that are spuriously correlated with the class label. Finally, we personalize a text-to-image generation model using the edited images to generate diverse in-domain images without spurious features. ASPIRE is complementary to all prior robust training methods in literature, and we demonstrate its effectiveness across 4 datasets and 9 baselines and show that ASPIRE improves the worst-group classification accuracy of prior methods by 1% - 38%. We also contribute a novel test set for the challenging Hard ImageNet dataset. |
| title | ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlations |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2308.10103 |