Are Compressed Language Models Less Subgroup Robust?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913284917035008 |
|---|---|
| author | Gee, Leonidas Zugarini, Andrea Quadrianto, Novi |
| author_facet | Gee, Leonidas Zugarini, Andrea Quadrianto, Novi |
| contents | To reduce the inference cost of large language models, model compression is increasingly used to create smaller scalable models. However, little is known about their robustness to minority subgroups defined by the labels and attributes of a dataset. In this paper, we investigate the effects of 18 different compression methods and settings on the subgroup robustness of BERT language models. We show that worst-group performance does not depend on model size alone, but also on the compression method used. Additionally, we find that model compression does not always worsen the performance on minority subgroups. Altogether, our analysis serves to further research into the subgroup robustness of model compression. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17811 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are Compressed Language Models Less Subgroup Robust? Gee, Leonidas Zugarini, Andrea Quadrianto, Novi Machine Learning Computation and Language To reduce the inference cost of large language models, model compression is increasingly used to create smaller scalable models. However, little is known about their robustness to minority subgroups defined by the labels and attributes of a dataset. In this paper, we investigate the effects of 18 different compression methods and settings on the subgroup robustness of BERT language models. We show that worst-group performance does not depend on model size alone, but also on the compression method used. Additionally, we find that model compression does not always worsen the performance on minority subgroups. Altogether, our analysis serves to further research into the subgroup robustness of model compression. |
| title | Are Compressed Language Models Less Subgroup Robust? |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2403.17811 |