Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918105177915392 |
|---|---|
| author | Do, Timothy Saran, Pranav Poojary, Harshita Prabhu, Pranav O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| author_facet | Do, Timothy Saran, Pranav Poojary, Harshita Prabhu, Pranav O'Brien, Sean Sharma, Vasu Zhu, Kevin |
| contents | In this paper, we address the persistent challenges that figurative language expressions pose for natural language processing (NLP) systems, particularly in low-resource languages such as Konkani. We present a hybrid model that integrates a pre-trained Multilingual BERT (mBERT) with a bidirectional LSTM and a linear classifier. This architecture is fine-tuned on a newly introduced annotated dataset for metaphor classification, developed as part of this work. To improve the model's efficiency, we implement a gradient-based attention head pruning strategy. For metaphor classification, the pruned model achieves an accuracy of 78%. We also applied our pruning approach to expand on an existing idiom classification task, achieving 83% accuracy. These results demonstrate the effectiveness of attention head pruning for building efficient NLP tools in underrepresented languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02005 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT Do, Timothy Saran, Pranav Poojary, Harshita Prabhu, Pranav O'Brien, Sean Sharma, Vasu Zhu, Kevin Computation and Language In this paper, we address the persistent challenges that figurative language expressions pose for natural language processing (NLP) systems, particularly in low-resource languages such as Konkani. We present a hybrid model that integrates a pre-trained Multilingual BERT (mBERT) with a bidirectional LSTM and a linear classifier. This architecture is fine-tuned on a newly introduced annotated dataset for metaphor classification, developed as part of this work. To improve the model's efficiency, we implement a gradient-based attention head pruning strategy. For metaphor classification, the pruned model achieves an accuracy of 78%. We also applied our pruning approach to expand on an existing idiom classification task, achieving 83% accuracy. These results demonstrate the effectiveness of attention head pruning for building efficient NLP tools in underrepresented languages. |
| title | Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.02005 |