Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.17913 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908242994528256 |
|---|---|
| author | Dannemann, Joris Junike, Gero |
| author_facet | Dannemann, Joris Junike, Gero |
| contents | Batch normalization is one of the most important regularization techniques for neural networks, significantly improving training by centering the layers of the neural network. There have been several attempts to provide a theoretical justification for batch ormalization. Santurkar and Tsipras (2018) [How does batch normalization help optimization? Advances in neural information rocessing systems, 31] claim that batch normalization improves initialization. We provide a counterexample showing that this claim s not true, i.e., batch normalization does not improve initialization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_17913 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Batch normalization does not improve initialization Dannemann, Joris Junike, Gero Machine Learning Probability Batch normalization is one of the most important regularization techniques for neural networks, significantly improving training by centering the layers of the neural network. There have been several attempts to provide a theoretical justification for batch ormalization. Santurkar and Tsipras (2018) [How does batch normalization help optimization? Advances in neural information rocessing systems, 31] claim that batch normalization improves initialization. We provide a counterexample showing that this claim s not true, i.e., batch normalization does not improve initialization. |
| title | Batch normalization does not improve initialization |
| topic | Machine Learning Probability |
| url | https://arxiv.org/abs/2502.17913 |