Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.00371 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914852562272256 |
|---|---|
| author | Zhou, Linjiang Shi, Xiaochuan Ma, Chao Wang, Zepeng |
| author_facet | Zhou, Linjiang Shi, Xiaochuan Ma, Chao Wang, Zepeng |
| contents | Gradients play a pivotal role in neural networks explanation. The inherent high dimensionality and structural complexity of neural networks result in the original gradients containing a significant amount of noise. While several approaches were proposed to reduce noise with smoothing, there is little discussion of the rationale behind smoothing gradients in neural networks. In this work, we proposed a gradient smooth theoretical framework for neural networks based on the function mollification and Monte Carlo integration. The framework intrinsically axiomatized gradient smoothing and reveals the rationale of existing methods. Furthermore, we provided an approach to design new smooth methods derived from the framework. By experimental measurement of several newly designed smooth methods, we demonstrated the research potential of our framework. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_00371 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Axiomatization of Gradient Smoothing in Neural Networks Zhou, Linjiang Shi, Xiaochuan Ma, Chao Wang, Zepeng Machine Learning Artificial Intelligence Gradients play a pivotal role in neural networks explanation. The inherent high dimensionality and structural complexity of neural networks result in the original gradients containing a significant amount of noise. While several approaches were proposed to reduce noise with smoothing, there is little discussion of the rationale behind smoothing gradients in neural networks. In this work, we proposed a gradient smooth theoretical framework for neural networks based on the function mollification and Monte Carlo integration. The framework intrinsically axiomatized gradient smoothing and reveals the rationale of existing methods. Furthermore, we provided an approach to design new smooth methods derived from the framework. By experimental measurement of several newly designed smooth methods, we demonstrated the research potential of our framework. |
| title | Axiomatization of Gradient Smoothing in Neural Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2407.00371 |