"Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909615440003072 |
|---|---|
| author | Liang, Zi Ye, Qingqing Wang, Yanyun Zhang, Sen Xiao, Yaxin Li, Ronghua Xu, Jianliang Hu, Haibo |
| author_facet | Liang, Zi Ye, Qingqing Wang, Yanyun Zhang, Sen Xiao, Yaxin Li, Ronghua Xu, Jianliang Hu, Haibo |
| contents | Model extraction attacks (MEAs) on large language models (LLMs) have received increasing attention in recent research. However, existing attack methods typically adapt the extraction strategies originally developed for deep neural networks (DNNs). They neglect the underlying inconsistency between the training tasks of MEA and LLM alignment, leading to suboptimal attack performance. To tackle this issue, we propose Locality Reinforced Distillation (LoRD), a novel model extraction algorithm specifically designed for LLMs. In particular, LoRD employs a newly defined policy-gradient-style training task that utilizes the responses of victim model as the signal to guide the crafting of preference for the local model. Theoretical analyses demonstrate that I) The convergence procedure of LoRD in model extraction is consistent with the alignment procedure of LLMs, and II) LoRD can reduce query complexity while mitigating watermark protection through our exploration-based stealing. Extensive experiments validate the superiority of our method in extracting various state-of-the-art commercial LLMs. Our code is available at: https://github.com/liangzid/LoRD-MEA . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_02718 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | "Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation Liang, Zi Ye, Qingqing Wang, Yanyun Zhang, Sen Xiao, Yaxin Li, Ronghua Xu, Jianliang Hu, Haibo Cryptography and Security Computation and Language Model extraction attacks (MEAs) on large language models (LLMs) have received increasing attention in recent research. However, existing attack methods typically adapt the extraction strategies originally developed for deep neural networks (DNNs). They neglect the underlying inconsistency between the training tasks of MEA and LLM alignment, leading to suboptimal attack performance. To tackle this issue, we propose Locality Reinforced Distillation (LoRD), a novel model extraction algorithm specifically designed for LLMs. In particular, LoRD employs a newly defined policy-gradient-style training task that utilizes the responses of victim model as the signal to guide the crafting of preference for the local model. Theoretical analyses demonstrate that I) The convergence procedure of LoRD in model extraction is consistent with the alignment procedure of LLMs, and II) LoRD can reduce query complexity while mitigating watermark protection through our exploration-based stealing. Extensive experiments validate the superiority of our method in extracting various state-of-the-art commercial LLMs. Our code is available at: https://github.com/liangzid/LoRD-MEA . |
| title | "Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillation |
| topic | Cryptography and Security Computation and Language |
| url | https://arxiv.org/abs/2409.02718 |