Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918062061518848 |
|---|---|
| author | Bouaziz, Wassim Videau, Mathurin Usunier, Nicolas El-Mhamdi, El-Mahdi |
| author_facet | Bouaziz, Wassim Videau, Mathurin Usunier, Nicolas El-Mhamdi, El-Mahdi |
| contents | The pre-training of large language models (LLMs) relies on massive text datasets sourced from diverse and difficult-to-curate origins. Although membership inference attacks and hidden canaries have been explored to trace data usage, such methods rely on memorization of training data, which LM providers try to limit. In this work, we demonstrate that indirect data poisoning (where the targeted behavior is absent from training data) is not only feasible but also allow to effectively protect a dataset and trace its use. Using gradient-based optimization prompt-tuning, we make a model learn arbitrary secret sequences: secret responses to secret prompts that are absent from the training corpus. We validate our approach on language models pre-trained from scratch and show that less than 0.005% of poisoned tokens are sufficient to covertly make a LM learn a secret and detect it with extremely high confidence ($p < 10^{-55}$) with a theoretically certifiable scheme. Crucially, this occurs without performance degradation (on LM benchmarks) and despite secrets never appearing in the training set. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14913 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning Bouaziz, Wassim Videau, Mathurin Usunier, Nicolas El-Mhamdi, El-Mahdi Cryptography and Security Machine Learning The pre-training of large language models (LLMs) relies on massive text datasets sourced from diverse and difficult-to-curate origins. Although membership inference attacks and hidden canaries have been explored to trace data usage, such methods rely on memorization of training data, which LM providers try to limit. In this work, we demonstrate that indirect data poisoning (where the targeted behavior is absent from training data) is not only feasible but also allow to effectively protect a dataset and trace its use. Using gradient-based optimization prompt-tuning, we make a model learn arbitrary secret sequences: secret responses to secret prompts that are absent from the training corpus. We validate our approach on language models pre-trained from scratch and show that less than 0.005% of poisoned tokens are sufficient to covertly make a LM learn a secret and detect it with extremely high confidence ($p < 10^{-55}$) with a theoretically certifiable scheme. Crucially, this occurs without performance degradation (on LM benchmarks) and despite secrets never appearing in the training set. |
| title | Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2506.14913 |