Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913434469138432 |
|---|---|
| author | Sinha, Aman Mickus, Timothee Clausel, Marianne Constant, Mathieu Coubez, Xavier |
| author_facet | Sinha, Aman Mickus, Timothee Clausel, Marianne Constant, Mathieu Coubez, Xavier |
| contents | The success of pretrained language models (PLMs) across a spate of use-cases has led to significant investment from the NLP community towards building domain-specific foundational models. On the other hand, in mission critical settings such as biomedical applications, other aspects also factor in-chief of which is a model's ability to produce reasonable estimates of its own uncertainty. In the present study, we discuss these two desiderata through the lens of how they shape the entropy of a model's output probability distribution. We find that domain specificity and uncertainty awareness can often be successfully combined, but the exact task at hand weighs in much more strongly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12626 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification? Sinha, Aman Mickus, Timothee Clausel, Marianne Constant, Mathieu Coubez, Xavier Computation and Language The success of pretrained language models (PLMs) across a spate of use-cases has led to significant investment from the NLP community towards building domain-specific foundational models. On the other hand, in mission critical settings such as biomedical applications, other aspects also factor in-chief of which is a model's ability to produce reasonable estimates of its own uncertainty. In the present study, we discuss these two desiderata through the lens of how they shape the entropy of a model's output probability distribution. We find that domain specificity and uncertainty awareness can often be successfully combined, but the exact task at hand weighs in much more strongly. |
| title | Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2407.12626 |