Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915148487196672 |
|---|---|
| author | Zhang, Junpeng Cheng, Lei Li, Qing Lin, Liang Zhang, Quanshi |
| author_facet | Zhang, Junpeng Cheng, Lei Li, Qing Lin, Liang Zhang, Quanshi |
| contents | In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples of a DNN, which are represented by non-generalizable interactions, are determined by its low-layer parameters. In comparison, other factors, such as high-layer parameters and network architecture, have much less impact on the composition of confusing samples. Two DNNs with different low-layer parameters usually have fully different sets of confusing samples, even though they have similar performance. This finding extends the understanding of the lottery ticket hypothesis, and well explains distinctive representation power of different DNNs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_08625 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN Zhang, Junpeng Cheng, Lei Li, Qing Lin, Liang Zhang, Quanshi Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples of a DNN, which are represented by non-generalizable interactions, are determined by its low-layer parameters. In comparison, other factors, such as high-layer parameters and network architecture, have much less impact on the composition of confusing samples. Two DNNs with different low-layer parameters usually have fully different sets of confusing samples, even though they have similar performance. This finding extends the understanding of the lottery ticket hypothesis, and well explains distinctive representation power of different DNNs. |
| title | Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN |
| topic | Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.08625 |