Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915876212572160 |
|---|---|
| author | Hou, Iyad Ait Borad, Shrenik Sharma, Harsh Srinivasan, Pooja Hwa, Rebecca Zirikly, Aya |
| author_facet | Hou, Iyad Ait Borad, Shrenik Sharma, Harsh Srinivasan, Pooja Hwa, Rebecca Zirikly, Aya |
| contents | Machine unlearning is increasingly important for clinical language models, where privacy regulations and institutional policies may require removing sensitive information from deployed systems without retraining from scratch. In practice, deletion requests must balance effective forgetting of targeted information with preservation of model utility and minimal parameter modification. We introduce Sparse Token Embedding Unlearning (STEU), a parameter-efficient method for behavioral class-level unlearning that updates only PMI-selected token embeddings together with a small classifier head while keeping all encoder layers frozen. Across experiments on MIMIC-IV, MIMIC-III, and eICU using BioClinicalBERT, BERT-base, and DistilBERT, STEU consistently suppresses the target class while largely preserving retained task performance. In the primary MIMIC-IV setting, STEU achieves near-complete forgetting (forget F1 = 0.0004) while maintaining competitive retained utility (retain avg F1 = 0.4766) after modifying only 0.19\% of model parameters. These results suggest that targeted behavioral unlearning can be achieved through sparse embedding edits without modifying deeper encoder representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19302 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning Hou, Iyad Ait Borad, Shrenik Sharma, Harsh Srinivasan, Pooja Hwa, Rebecca Zirikly, Aya Machine Learning Artificial Intelligence Machine unlearning is increasingly important for clinical language models, where privacy regulations and institutional policies may require removing sensitive information from deployed systems without retraining from scratch. In practice, deletion requests must balance effective forgetting of targeted information with preservation of model utility and minimal parameter modification. We introduce Sparse Token Embedding Unlearning (STEU), a parameter-efficient method for behavioral class-level unlearning that updates only PMI-selected token embeddings together with a small classifier head while keeping all encoder layers frozen. Across experiments on MIMIC-IV, MIMIC-III, and eICU using BioClinicalBERT, BERT-base, and DistilBERT, STEU consistently suppresses the target class while largely preserving retained task performance. In the primary MIMIC-IV setting, STEU achieves near-complete forgetting (forget F1 = 0.0004) while maintaining competitive retained utility (retain avg F1 = 0.4766) after modifying only 0.19\% of model parameters. These results suggest that targeted behavioral unlearning can be achieved through sparse embedding edits without modifying deeper encoder representations. |
| title | Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2603.19302 |