Persistent Pre-Training Poisoning of LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912075496816640 |
|---|---|
| author | Zhang, Yiming Rando, Javier Evtimov, Ivan Chi, Jianfeng Smith, Eric Michael Carlini, Nicholas Tramèr, Florian Ippolito, Daphne |
| author_facet | Zhang, Yiming Rando, Javier Evtimov, Ivan Chi, Jianfeng Smith, Eric Michael Carlini, Nicholas Tramèr, Florian Ippolito, Daphne |
| contents | Large language models are pre-trained on uncurated text datasets consisting of trillions of tokens scraped from the Web. Prior work has shown that: (1) web-scraped pre-training datasets can be practically poisoned by malicious actors; and (2) adversaries can compromise language models after poisoning fine-tuning datasets. Our work evaluates for the first time whether language models can also be compromised during pre-training, with a focus on the persistence of pre-training attacks after models are fine-tuned as helpful and harmless chatbots (i.e., after SFT and DPO). We pre-train a series of LLMs from scratch to measure the impact of a potential poisoning adversary under four different attack objectives (denial-of-service, belief manipulation, jailbreaking, and prompt stealing), and across a wide range of model sizes (from 600M to 7B). Our main result is that poisoning only 0.1% of a model's pre-training dataset is sufficient for three out of four attacks to measurably persist through post-training. Moreover, simple attacks like denial-of-service persist through post-training with a poisoning rate of only 0.001%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13722 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Persistent Pre-Training Poisoning of LLMs Zhang, Yiming Rando, Javier Evtimov, Ivan Chi, Jianfeng Smith, Eric Michael Carlini, Nicholas Tramèr, Florian Ippolito, Daphne Cryptography and Security Artificial Intelligence Large language models are pre-trained on uncurated text datasets consisting of trillions of tokens scraped from the Web. Prior work has shown that: (1) web-scraped pre-training datasets can be practically poisoned by malicious actors; and (2) adversaries can compromise language models after poisoning fine-tuning datasets. Our work evaluates for the first time whether language models can also be compromised during pre-training, with a focus on the persistence of pre-training attacks after models are fine-tuned as helpful and harmless chatbots (i.e., after SFT and DPO). We pre-train a series of LLMs from scratch to measure the impact of a potential poisoning adversary under four different attack objectives (denial-of-service, belief manipulation, jailbreaking, and prompt stealing), and across a wide range of model sizes (from 600M to 7B). Our main result is that poisoning only 0.1% of a model's pre-training dataset is sufficient for three out of four attacks to measurably persist through post-training. Moreover, simple attacks like denial-of-service persist through post-training with a poisoning rate of only 0.001%. |
| title | Persistent Pre-Training Poisoning of LLMs |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2410.13722 |