Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognition
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917769306439680 |
|---|---|
| author | Rehman, Abdul Zhang, Jian Jun Yang, Xiaosong |
| author_facet | Rehman, Abdul Zhang, Jian Jun Yang, Xiaosong |
| contents | Named Entity Recognition (NER) encounters the challenge of unbalanced labels, where certain entity types are overrepresented while others are underrepresented in real-world datasets. This imbalance can lead to biased models that perform poorly on minority entity classes, impeding accurate and equitable entity recognition. This paper explores the effects of unbalanced entity labels of the BERT-based pre-trained model. We analyze the different mechanisms of loss calculation and loss propagation for the task of token classification on randomized datasets. Then we propose ways to improve the token classification for the highly imbalanced task of clinical entity recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_03238 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognition Rehman, Abdul Zhang, Jian Jun Yang, Xiaosong Computation and Language Machine Learning 68T50 I.2.7 Named Entity Recognition (NER) encounters the challenge of unbalanced labels, where certain entity types are overrepresented while others are underrepresented in real-world datasets. This imbalance can lead to biased models that perform poorly on minority entity classes, impeding accurate and equitable entity recognition. This paper explores the effects of unbalanced entity labels of the BERT-based pre-trained model. We analyze the different mechanisms of loss calculation and loss propagation for the task of token classification on randomized datasets. Then we propose ways to improve the token classification for the highly imbalanced task of clinical entity recognition. |
| title | Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognition |
| topic | Computation and Language Machine Learning 68T50 I.2.7 |
| url | https://arxiv.org/abs/2409.03238 |