Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929406515085312 |
|---|---|
| author | MacPhail, Dorothea Harbecke, David Raithel, Lisa Möller, Sebastian |
| author_facet | MacPhail, Dorothea Harbecke, David Raithel, Lisa Möller, Sebastian |
| contents | An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02432 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates MacPhail, Dorothea Harbecke, David Raithel, Lisa Möller, Sebastian Computation and Language Machine Learning An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities. |
| title | Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2407.02432 |