Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.11737 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917592921276416 |
|---|---|
| author | Mo, Zihao Yang, Yejiang Lu, Shuaizheng Xiang, Weiming |
| author_facet | Mo, Zihao Yang, Yejiang Lu, Shuaizheng Xiang, Weiming |
| contents | In this paper, we propose a method of repairing compressed Feedforward Neural Networks (FNNs) based on equivalence evaluation of two neural networks. In the repairing framework, a novel neural network equivalence evaluation method is developed to compute the output discrepancy between two neural networks. The output discrepancy can quantitatively characterize the output difference produced by compression procedures. Based on the computed output discrepancy, the repairing method first initializes a new training set for the compressed networks to narrow down the discrepancy between the two neural networks and improve the performance of the compressed network. Then, we repair the compressed FNN by re-training based on the training set. We apply our developed method to the MNIST dataset to demonstrate the effectiveness and advantages of our proposed repair method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11737 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Compression Repair for Feedforward Neural Networks Based on Model Equivalence Evaluation Mo, Zihao Yang, Yejiang Lu, Shuaizheng Xiang, Weiming Machine Learning Artificial Intelligence In this paper, we propose a method of repairing compressed Feedforward Neural Networks (FNNs) based on equivalence evaluation of two neural networks. In the repairing framework, a novel neural network equivalence evaluation method is developed to compute the output discrepancy between two neural networks. The output discrepancy can quantitatively characterize the output difference produced by compression procedures. Based on the computed output discrepancy, the repairing method first initializes a new training set for the compressed networks to narrow down the discrepancy between the two neural networks and improve the performance of the compressed network. Then, we repair the compressed FNN by re-training based on the training set. We apply our developed method to the MNIST dataset to demonstrate the effectiveness and advantages of our proposed repair method. |
| title | Compression Repair for Feedforward Neural Networks Based on Model Equivalence Evaluation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2402.11737 |