Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916483742826496 |
|---|---|
| author | Singh, Priyanka Mosin, Vladislav D. Yamshchikov, Ivan P. |
| author_facet | Singh, Priyanka Mosin, Vladislav D. Yamshchikov, Ivan P. |
| contents | Working within specific NLP subdomains presents significant challenges, primarily due to a persistent deficit of data. Stringent privacy concerns and limited data accessibility often drive this shortage. Additionally, the medical domain demands high accuracy, where even marginal improvements in model performance can have profound impacts. In this study, we investigate the potential of vocabulary transfer to enhance model performance in biomedical NLP tasks. Specifically, we focus on vocabulary extension, a technique that involves expanding the target vocabulary to incorporate domain-specific biomedical terms. Our findings demonstrate that vocabulary extension, leads to measurable improvements in both downstream model performance and inference time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2208_02554 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data Singh, Priyanka Mosin, Vladislav D. Yamshchikov, Ivan P. Computation and Language I.2.7 Working within specific NLP subdomains presents significant challenges, primarily due to a persistent deficit of data. Stringent privacy concerns and limited data accessibility often drive this shortage. Additionally, the medical domain demands high accuracy, where even marginal improvements in model performance can have profound impacts. In this study, we investigate the potential of vocabulary transfer to enhance model performance in biomedical NLP tasks. Specifically, we focus on vocabulary extension, a technique that involves expanding the target vocabulary to incorporate domain-specific biomedical terms. Our findings demonstrate that vocabulary extension, leads to measurable improvements in both downstream model performance and inference time. |
| title | Vocabulary Transfer for Biomedical Texts: Add Tokens if You Can Not Add Data |
| topic | Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2208.02554 |