Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.06633 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917689898827776 |
|---|---|
| author | Qiu, Xiaoqi Wang, Yongjie Guo, Xu Zeng, Zhiwei Yu, Yue Feng, Yuhong Miao, Chunyan |
| author_facet | Qiu, Xiaoqi Wang, Yongjie Guo, Xu Zeng, Zhiwei Yu, Yue Feng, Yuhong Miao, Chunyan |
| contents | Counterfactually Augmented Data (CAD) involves creating new data samples by applying minimal yet sufficient modifications to flip the label of existing data samples to other classes. Training with CAD enhances model robustness against spurious features that happen to correlate with labels by spreading the casual relationships across different classes. Yet, recent research reveals that training with CAD may lead models to overly focus on modified features while ignoring other important contextual information, inadvertently introducing biases that may impair performance on out-ofdistribution (OOD) datasets. To mitigate this issue, we employ contrastive learning to promote global feature alignment in addition to learning counterfactual clues. We theoretically prove that contrastive loss can encourage models to leverage a broader range of features beyond those modified ones. Comprehensive experiments on two human-edited CAD datasets demonstrate that our proposed method outperforms the state-of-the-art on OOD datasets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_06633 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning Qiu, Xiaoqi Wang, Yongjie Guo, Xu Zeng, Zhiwei Yu, Yue Feng, Yuhong Miao, Chunyan Machine Learning 68T50 I.2; I.2.7 Counterfactually Augmented Data (CAD) involves creating new data samples by applying minimal yet sufficient modifications to flip the label of existing data samples to other classes. Training with CAD enhances model robustness against spurious features that happen to correlate with labels by spreading the casual relationships across different classes. Yet, recent research reveals that training with CAD may lead models to overly focus on modified features while ignoring other important contextual information, inadvertently introducing biases that may impair performance on out-ofdistribution (OOD) datasets. To mitigate this issue, we employ contrastive learning to promote global feature alignment in addition to learning counterfactual clues. We theoretically prove that contrastive loss can encourage models to leverage a broader range of features beyond those modified ones. Comprehensive experiments on two human-edited CAD datasets demonstrate that our proposed method outperforms the state-of-the-art on OOD datasets. |
| title | PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning |
| topic | Machine Learning 68T50 I.2; I.2.7 |
| url | https://arxiv.org/abs/2406.06633 |