A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908884766031872 |
|---|---|
| author | Xin, Rui Mireshghallah, Niloofar Li, Shuyue Stella Duan, Michael Kim, Hyunwoo Choi, Yejin Tsvetkov, Yulia Oh, Sewoong Koh, Pang Wei |
| author_facet | Xin, Rui Mireshghallah, Niloofar Li, Shuyue Stella Duan, Michael Kim, Hyunwoo Choi, Yejin Tsvetkov, Yulia Oh, Sewoong Koh, Pang Wei |
| contents | Sanitizing sensitive text data typically involves removing personally identifiable information (PII) or generating synthetic data under the assumption that these methods adequately protect privacy; however, their effectiveness is often only assessed by measuring the leakage of explicit identifiers but ignoring nuanced textual markers that can lead to re-identification. We challenge the above illusion of privacy by proposing a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release. Our approach shows that seemingly innocuous auxiliary information -- such as routine social activities -- can be used to infer sensitive attributes like age or substance use history from sanitized data. For instance, we demonstrate that Azure's commercial PII removal tool fails to protect 74\% of information in the MedQA dataset. Although differential privacy mitigates these risks to some extent, it significantly reduces the utility of the sanitized text for downstream tasks. Our findings indicate that current sanitization techniques offer a \textit{false sense of privacy}, highlighting the need for more robust methods that protect against semantic-level information leakage. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_21035 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage Xin, Rui Mireshghallah, Niloofar Li, Shuyue Stella Duan, Michael Kim, Hyunwoo Choi, Yejin Tsvetkov, Yulia Oh, Sewoong Koh, Pang Wei Cryptography and Security Computation and Language Machine Learning Sanitizing sensitive text data typically involves removing personally identifiable information (PII) or generating synthetic data under the assumption that these methods adequately protect privacy; however, their effectiveness is often only assessed by measuring the leakage of explicit identifiers but ignoring nuanced textual markers that can lead to re-identification. We challenge the above illusion of privacy by proposing a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release. Our approach shows that seemingly innocuous auxiliary information -- such as routine social activities -- can be used to infer sensitive attributes like age or substance use history from sanitized data. For instance, we demonstrate that Azure's commercial PII removal tool fails to protect 74\% of information in the MedQA dataset. Although differential privacy mitigates these risks to some extent, it significantly reduces the utility of the sanitized text for downstream tasks. Our findings indicate that current sanitization techniques offer a \textit{false sense of privacy}, highlighting the need for more robust methods that protect against semantic-level information leakage. |
| title | A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage |
| topic | Cryptography and Security Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2504.21035 |