Scalable multilingual PII annotation for responsible AI in LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917000856469504 |
|---|---|
| author | Meena, Bharti Skubisz, Joanna Rajgarhia, Harshit Dave, Nand Ganesh, Kiran Dalmia, Shivali Mukherji, Abhishek Sundarababu, Vasudevan |
| author_facet | Meena, Bharti Skubisz, Joanna Rajgarhia, Harshit Dave, Nand Ganesh, Kiran Dalmia, Shivali Mukherji, Abhishek Sundarababu, Vasudevan |
| contents | As Large Language Models (LLMs) gain wider adoption, ensuring their reliable handling of Personally Identifiable Information (PII) across diverse regulatory contexts has become essential. This work introduces a scalable multilingual data curation framework designed for high-quality PII annotation across 13 underrepresented locales, covering approximately 336 locale-specific PII types. Our phased, human-in-the-loop annotation methodology combines linguistic expertise with rigorous quality assurance, leading to substantial improvements in recall and false positive rates from pilot, training, and production phases. By leveraging inter-annotator agreement metrics and root-cause analysis, the framework systematically uncovers and resolves annotation inconsistencies, resulting in high-fidelity datasets suitable for supervised LLM fine-tuning. Beyond reporting empirical gains, we highlight common annotator challenges in multilingual PII labeling and demonstrate how iterative, analytics-driven pipelines can enhance both annotation quality and downstream model reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06250 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scalable multilingual PII annotation for responsible AI in LLMs Meena, Bharti Skubisz, Joanna Rajgarhia, Harshit Dave, Nand Ganesh, Kiran Dalmia, Shivali Mukherji, Abhishek Sundarababu, Vasudevan Computation and Language Artificial Intelligence As Large Language Models (LLMs) gain wider adoption, ensuring their reliable handling of Personally Identifiable Information (PII) across diverse regulatory contexts has become essential. This work introduces a scalable multilingual data curation framework designed for high-quality PII annotation across 13 underrepresented locales, covering approximately 336 locale-specific PII types. Our phased, human-in-the-loop annotation methodology combines linguistic expertise with rigorous quality assurance, leading to substantial improvements in recall and false positive rates from pilot, training, and production phases. By leveraging inter-annotator agreement metrics and root-cause analysis, the framework systematically uncovers and resolves annotation inconsistencies, resulting in high-fidelity datasets suitable for supervised LLM fine-tuning. Beyond reporting empirical gains, we highlight common annotator challenges in multilingual PII labeling and demonstrate how iterative, analytics-driven pipelines can enhance both annotation quality and downstream model reliability. |
| title | Scalable multilingual PII annotation for responsible AI in LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.06250 |