Cover Image

On Importance of Layer Pruning for Smaller BERT Models and Low Resource Languages

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shirke, Mayur, Shembade, Amey, Wagh, Madhushri, Thorat, Pavan, Joshi, Raviraj
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Comparative Study of Pre-Trained BERT and Large Language Models for Code-Mixed Named Entity Recognition
by: Shirke, Mayur, et al.
Published: (2025)

Towards Building Efficient Sentence BERT Models using Layer Pruning
by: Shelke, Anushka, et al.
Published: (2024)

On Importance of Pruning and Distillation for Efficient Low Resource NLP
by: Mirashi, Aishwarya, et al.
Published: (2024)

Chain-of-Translation Prompting (CoTR): A Novel Prompting Technique for Low Resource Languages
by: Deshpande, Tejas, et al.
Published: (2024)

On Limitations of LLM as Annotator for Low Resource Languages
by: Jadhav, Suramya, et al.
Published: (2024)

Better To Ask in English? Evaluating Factual Accuracy of Multilingual LLMs in English and Low-Resource Languages
by: Rohera, Pritika, et al.
Published: (2025)

MahaParaphrase: A Marathi Paraphrase Detection Corpus and BERT-based Models
by: Jadhav, Suramya, et al.
Published: (2025)

A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations
by: Kowtal, Nidhi, et al.
Published: (2024)

Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning
by: Khade, Omkar, et al.
Published: (2024)

L3Cube-MahaEmotions: A Marathi Emotion Recognition Dataset with Synthetic Annotations using CoTR prompting and Large Language Models
by: Kowtal, Nidhi, et al.
Published: (2025)

Non-Contextual BERT or FastText? A Comparative Analysis
by: Shanbhag, Abhay, et al.
Published: (2024)

Topic Modeling in Marathi
by: Shinde, Sanket, et al.
Published: (2025)

L3Cube-IndicHeadline-ID: A Dataset for Headline Identification and Semantic Evaluation in Low-Resource Indian Languages
by: Tanksale, Nishant, et al.
Published: (2025)

Aligning Large Language Models to Low-Resource Languages through LLM-Based Selective Translation: A Systematic Study
by: Paul, Rakesh, et al.
Published: (2025)

On Importance of Code-Mixed Embeddings for Hate Speech Identification
by: Jagdale, Shruti, et al.
Published: (2024)

L3Cube-MahaSTS: A Marathi Sentence Similarity Dataset and Models
by: Mirashi, Aishwarya, et al.
Published: (2025)

Leveraging Parameter Efficient Training Methods for Low Resource Text Classification: A Case Study in Marathi
by: Deshmukh, Pranita, et al.
Published: (2024)

L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT models
by: Chaudhari, Harsh, et al.
Published: (2023)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

LakotaBERT: A Transformer-based Model for Low Resource Lakota Language
by: Parankusham, Kanishka, et al.
Published: (2025)

Adapting Multilingual LLMs to Low-Resource Languages using Continued Pre-training and Synthetic Corpus
by: Joshi, Raviraj, et al.
Published: (2024)

DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models
by: Thorat, Shantanu, et al.
Published: (2025)

Investigating Layer Importance in Large Language Models
by: Zhang, Yang, et al.
Published: (2024)

IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languages
by: Endait, Sharvi, et al.
Published: (2025)

L3Cube-IndicNews: News-based Short Text and Long Document Classification Datasets in Indic Languages
by: Mirashi, Aishwarya, et al.
Published: (2024)

Adaptive Pruning for Large Language Models with Structural Importance Awareness
by: Zheng, Haotian, et al.
Published: (2024)

Improving the Efficiency of Long Document Classification using Sentence Ranking Approach
by: Kokate, Prathamesh, et al.
Published: (2025)

Curating Stopwords in Marathi: A TF-IDF Approach for Improved Text Analysis and Information Retrieval
by: Chavan, Rohan, et al.
Published: (2024)

L3Cube-MahaSum: A Comprehensive Dataset and BART Models for Abstractive Text Summarization in Marathi
by: Deshmukh, Pranita, et al.
Published: (2024)

Universal Cross-Lingual Text Classification
by: Savant, Riya, et al.
Published: (2024)

Enhancing BERT Fine-Tuning for Sentiment Analysis in Lower-Resourced Languages
by: Kubík, Jozef, et al.
Published: (2025)

MaxPoolBERT: Enhancing BERT Classification via Layer- and Token-Wise Aggregation
by: Behrendt, Maike, et al.
Published: (2025)

HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs
by: Bazaga, Adrián, et al.
Published: (2024)

Efficient Zero-Shot Long Document Classification by Reducing Context Through Sentence Ranking
by: Kokate, Prathamesh, et al.
Published: (2025)

L3Cube-MahaNews: News-based Short Text and Long Document Classification Datasets in Marathi
by: Mittal, Saloni, et al.
Published: (2024)

MahaSQuAD: Bridging Linguistic Divides in Marathi Question-Answering
by: Ghatage, Ruturaj, et al.
Published: (2024)

L3Cube-IndicQuest: A Benchmark Question Answering Dataset for Evaluating Knowledge of LLMs in Indic Context
by: Rohera, Pritika, et al.
Published: (2024)

Adaptive LoRA Merge with Parameter Pruning for Low-Resource Generation
by: Miyano, Ryota, et al.
Published: (2025)

Smaller Language Models are Better Black-box Machine-Generated Text Detectors
by: Mireshghallah, Niloofar, et al.
Published: (2023)

Lost in State Space: Probing Frozen Mamba Representations
by: Wagh, Bhagyashree, et al.
Published: (2026)