Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913017652838400 |
|---|---|
| author | Nishikawa, Kohei Shimizu, Koki Hashiguchi, Hiroki |
| author_facet | Nishikawa, Kohei Shimizu, Koki Hashiguchi, Hiroki |
| contents | This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight matrix is modeled as the sum of signal and noise matrices. The low-rank approximation is obtained by removing noise-related singular values using a threshold based on random matrix theory. To assess the adequacy of this threshold, we propose an evaluation metric based on the cosine similarity between the singular vectors of the signal and original weight matrices. The proposed metric is used in numerical experiments to compare two threshold estimation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_12911 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory Nishikawa, Kohei Shimizu, Koki Hashiguchi, Hiroki Machine Learning This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight matrix is modeled as the sum of signal and noise matrices. The low-rank approximation is obtained by removing noise-related singular values using a threshold based on random matrix theory. To assess the adequacy of this threshold, we propose an evaluation metric based on the cosine similarity between the singular vectors of the signal and original weight matrices. The proposed metric is used in numerical experiments to compare two threshold estimation methods. |
| title | Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.12911 |