L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929243047329792 |
|---|---|
| author | Chaudhari, Harsh Patil, Anuja Lavekar, Dhanashree Khairnar, Pranav Joshi, Raviraj |
| author_facet | Chaudhari, Harsh Patil, Anuja Lavekar, Dhanashree Khairnar, Pranav Joshi, Raviraj |
| contents | This work introduces the L3Cube-MahaSocialNER dataset, the first and largest social media dataset specifically designed for Named Entity Recognition (NER) in the Marathi language. The dataset comprises 18,000 manually labeled sentences covering eight entity classes, addressing challenges posed by social media data, including non-standard language and informal idioms. Deep learning models, including CNN, LSTM, BiLSTM, and Transformer models, are evaluated on the individual dataset with IOB and non-IOB notations. The results demonstrate the effectiveness of these models in accurately recognizing named entities in Marathi informal text. The L3Cube-MahaSocialNER dataset offers user-centric information extraction and supports real-time applications, providing a valuable resource for public opinion analysis, news, and marketing on social media platforms. We also show that the zero-shot results of the regular NER model are poor on the social NER test set thus highlighting the need for more social NER datasets. The datasets and models are publicly available at https://github.com/l3cube-pune/MarathiNLP |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_00170 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT models Chaudhari, Harsh Patil, Anuja Lavekar, Dhanashree Khairnar, Pranav Joshi, Raviraj Computation and Language Machine Learning This work introduces the L3Cube-MahaSocialNER dataset, the first and largest social media dataset specifically designed for Named Entity Recognition (NER) in the Marathi language. The dataset comprises 18,000 manually labeled sentences covering eight entity classes, addressing challenges posed by social media data, including non-standard language and informal idioms. Deep learning models, including CNN, LSTM, BiLSTM, and Transformer models, are evaluated on the individual dataset with IOB and non-IOB notations. The results demonstrate the effectiveness of these models in accurately recognizing named entities in Marathi informal text. The L3Cube-MahaSocialNER dataset offers user-centric information extraction and supports real-time applications, providing a valuable resource for public opinion analysis, news, and marketing on social media platforms. We also show that the zero-shot results of the regular NER model are poor on the social NER test set thus highlighting the need for more social NER datasets. The datasets and models are publicly available at https://github.com/l3cube-pune/MarathiNLP |
| title | L3Cube-MahaSocialNER: A Social Media based Marathi NER Dataset and BERT models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2401.00170 |