Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2310.20673 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929268351565824 |
|---|---|
| author | Hashemizadeh, Meraj Ramirez, Juan Sukumaran, Rohan Farnadi, Golnoosh Lacoste-Julien, Simon Gallego-Posada, Jose |
| author_facet | Hashemizadeh, Meraj Ramirez, Juan Sukumaran, Rohan Farnadi, Golnoosh Lacoste-Julien, Simon Gallego-Posada, Jose |
| contents | Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse models achieve performance comparable to that of their dense counterparts at the level of the entire dataset, they exhibit high accuracy drops for some data sub-groups. Existing methods to mitigate this disparate impact induced by pruning (i) rely on surrogate metrics that address the problem indirectly and have limited interpretability; or (ii) scale poorly with the number of protected sub-groups in terms of computational cost. We propose a constrained optimization approach that directly addresses the disparate impact of pruning: our formulation bounds the accuracy change between the dense and sparse models, for each sub-group. This choice of constraints provides an interpretable success criterion to determine if a pruned model achieves acceptable disparity levels. Experimental results demonstrate that our technique scales reliably to problems involving large models and hundreds of protected sub-groups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_20673 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Balancing Act: Constraining Disparate Impact in Sparse Models Hashemizadeh, Meraj Ramirez, Juan Sukumaran, Rohan Farnadi, Golnoosh Lacoste-Julien, Simon Gallego-Posada, Jose Machine Learning Computers and Society Model pruning is a popular approach to enable the deployment of large deep learning models on edge devices with restricted computational or storage capacities. Although sparse models achieve performance comparable to that of their dense counterparts at the level of the entire dataset, they exhibit high accuracy drops for some data sub-groups. Existing methods to mitigate this disparate impact induced by pruning (i) rely on surrogate metrics that address the problem indirectly and have limited interpretability; or (ii) scale poorly with the number of protected sub-groups in terms of computational cost. We propose a constrained optimization approach that directly addresses the disparate impact of pruning: our formulation bounds the accuracy change between the dense and sparse models, for each sub-group. This choice of constraints provides an interpretable success criterion to determine if a pruned model achieves acceptable disparity levels. Experimental results demonstrate that our technique scales reliably to problems involving large models and hundreds of protected sub-groups. |
| title | Balancing Act: Constraining Disparate Impact in Sparse Models |
| topic | Machine Learning Computers and Society |
| url | https://arxiv.org/abs/2310.20673 |