Exploring explicit coarse-grained structure in artificial neural networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911921096097792 |
|---|---|
| author | Yang, Xi-Ci Xie, Z. Y. Yang, Xiao-Tao |
| author_facet | Yang, Xi-Ci Xie, Z. Y. Yang, Xiao-Tao |
| contents | We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_01779 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Exploring explicit coarse-grained structure in artificial neural networks Yang, Xi-Ci Xie, Z. Y. Yang, Xiao-Tao Machine Learning Statistical Mechanics Strongly Correlated Electrons Computer Vision and Pattern Recognition Quantum Physics We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. The idea has been applied in two situations. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly, without resorting to any magic nonlinear activations. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data that possesses the relevant features of the original dataset and can be used as references for classification. In both cases, the coarse-grained structure plays an important role in simplifying the network and improving both the interpretability and efficiency. The validity has been demonstrated on MNIST and CIFAR-10 datasets. Further improvement and some open questions related are also discussed. |
| title | Exploring explicit coarse-grained structure in artificial neural networks |
| topic | Machine Learning Statistical Mechanics Strongly Correlated Electrons Computer Vision and Pattern Recognition Quantum Physics |
| url | https://arxiv.org/abs/2211.01779 |