Restoring Rhythm: Punctuation Restoration Using Transformer Models for Bangla, A Low-Resource Language
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909987143417856 |
|---|---|
| author | Mamun, Md Obyedullahil Mamun, Md Adyelullahil Ahmad, Arif Emu, Md. Imran Hossain |
| author_facet | Mamun, Md Obyedullahil Mamun, Md Adyelullahil Ahmad, Arif Emu, Md. Imran Hossain |
| contents | Punctuation restoration enhances the readability of text and is critical for post-processing tasks in Automatic Speech Recognition (ASR), especially for low-resource languages like Bangla. In this study, we explore the application of transformer-based models, specifically XLM-RoBERTa-large, to automatically restore punctuation in unpunctuated Bangla text. We focus on predicting four punctuation marks: period, comma, question mark, and exclamation mark across diverse text domains. To address the scarcity of annotated resources, we constructed a large, varied training corpus and applied data augmentation techniques. Our best-performing model, trained with an augmentation factor of alpha = 0.20%, achieves an accuracy of 97.1% on the News test set, 91.2% on the Reference set, and 90.2% on the ASR set.
Results show strong generalization to reference and ASR transcripts, demonstrating the model's effectiveness in real-world, noisy scenarios. This work establishes a strong baseline for Bangla punctuation restoration and contributes publicly available datasets and code to support future research in low-resource NLP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18448 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Restoring Rhythm: Punctuation Restoration Using Transformer Models for Bangla, A Low-Resource Language Mamun, Md Obyedullahil Mamun, Md Adyelullahil Ahmad, Arif Emu, Md. Imran Hossain Computation and Language Artificial Intelligence Machine Learning I.2; I.7 Punctuation restoration enhances the readability of text and is critical for post-processing tasks in Automatic Speech Recognition (ASR), especially for low-resource languages like Bangla. In this study, we explore the application of transformer-based models, specifically XLM-RoBERTa-large, to automatically restore punctuation in unpunctuated Bangla text. We focus on predicting four punctuation marks: period, comma, question mark, and exclamation mark across diverse text domains. To address the scarcity of annotated resources, we constructed a large, varied training corpus and applied data augmentation techniques. Our best-performing model, trained with an augmentation factor of alpha = 0.20%, achieves an accuracy of 97.1% on the News test set, 91.2% on the Reference set, and 90.2% on the ASR set. Results show strong generalization to reference and ASR transcripts, demonstrating the model's effectiveness in real-world, noisy scenarios. This work establishes a strong baseline for Bangla punctuation restoration and contributes publicly available datasets and code to support future research in low-resource NLP. |
| title | Restoring Rhythm: Punctuation Restoration Using Transformer Models for Bangla, A Low-Resource Language |
| topic | Computation and Language Artificial Intelligence Machine Learning I.2; I.7 |
| url | https://arxiv.org/abs/2507.18448 |