Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.10723 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909495740858368 |
|---|---|
| author | Yang, Xiliang Deng, Shenyang Liu, Shicong Suo, Yuanchi NG, Wing. W. Y Zhang, Jianjun |
| author_facet | Yang, Xiliang Deng, Shenyang Liu, Shicong Suo, Yuanchi NG, Wing. W. Y Zhang, Jianjun |
| contents | Data augmentation is an important technique in training deep neural networks as it enhances their ability to generalize and remain robust. While data augmentation is commonly used to expand the sample size and act as a consistency regularization term, there is a lack of research on the relationship between them. To address this gap, this paper introduces a more comprehensive mathematical framework for data augmentation. Through this framework, we establish that the expected risk of the shifted population is the sum of the original population risk and a gap term, which can be interpreted as a consistency regularization term. The paper also provides a theoretical understanding of this gap, highlighting its negative effects on the early stages of training. We also propose a method to mitigate these effects. To validate our approach, we conducted experiments using same data augmentation techniques and computing resources under several scenarios, including standard training, out-of-distribution, and imbalanced classification. The results demonstrate that our methods surpass compared methods under all scenarios in terms of generalization ability and convergence stability. We provide our code implementation at the following link: https://github.com/ydlsfhll/ASPR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_10723 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Mathematics Framework of Artificial Shifted Population Risk and Its Further Understanding Related to Consistency Regularization Yang, Xiliang Deng, Shenyang Liu, Shicong Suo, Yuanchi NG, Wing. W. Y Zhang, Jianjun Machine Learning Artificial Intelligence Data augmentation is an important technique in training deep neural networks as it enhances their ability to generalize and remain robust. While data augmentation is commonly used to expand the sample size and act as a consistency regularization term, there is a lack of research on the relationship between them. To address this gap, this paper introduces a more comprehensive mathematical framework for data augmentation. Through this framework, we establish that the expected risk of the shifted population is the sum of the original population risk and a gap term, which can be interpreted as a consistency regularization term. The paper also provides a theoretical understanding of this gap, highlighting its negative effects on the early stages of training. We also propose a method to mitigate these effects. To validate our approach, we conducted experiments using same data augmentation techniques and computing resources under several scenarios, including standard training, out-of-distribution, and imbalanced classification. The results demonstrate that our methods surpass compared methods under all scenarios in terms of generalization ability and convergence stability. We provide our code implementation at the following link: https://github.com/ydlsfhll/ASPR. |
| title | A Mathematics Framework of Artificial Shifted Population Risk and Its Further Understanding Related to Consistency Regularization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.10723 |