UTF:Undertrained Tokens as Fingerprints A Novel Approach to LLM Identification
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915384598200320 |
|---|---|
| author | Cai, Jiacheng Yu, Jiahao Shao, Yangguang Wu, Yuhang |
| author_facet | Cai, Jiacheng Yu, Jiahao Shao, Yangguang Wu, Yuhang |
| contents | Fingerprinting large language models (LLMs) is essential for verifying model ownership, ensuring authenticity, and preventing misuse. Traditional fingerprinting methods often require significant computational overhead or white-box verification access. In this paper, we introduce UTF, a novel and efficient approach to fingerprinting LLMs by leveraging under-trained tokens. Under-trained tokens are tokens that the model has not fully learned during its training phase. By utilizing these tokens, we perform supervised fine-tuning to embed specific input-output pairs into the model. This process allows the LLM to produce predetermined outputs when presented with certain inputs, effectively embedding a unique fingerprint. Our method has minimal overhead and impact on model's performance, and does not require white-box access to target model's ownership identification. Compared to existing fingerprinting methods, UTF is also more effective and robust to fine-tuning and random guess. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12318 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | UTF:Undertrained Tokens as Fingerprints A Novel Approach to LLM Identification Cai, Jiacheng Yu, Jiahao Shao, Yangguang Wu, Yuhang Cryptography and Security Artificial Intelligence Fingerprinting large language models (LLMs) is essential for verifying model ownership, ensuring authenticity, and preventing misuse. Traditional fingerprinting methods often require significant computational overhead or white-box verification access. In this paper, we introduce UTF, a novel and efficient approach to fingerprinting LLMs by leveraging under-trained tokens. Under-trained tokens are tokens that the model has not fully learned during its training phase. By utilizing these tokens, we perform supervised fine-tuning to embed specific input-output pairs into the model. This process allows the LLM to produce predetermined outputs when presented with certain inputs, effectively embedding a unique fingerprint. Our method has minimal overhead and impact on model's performance, and does not require white-box access to target model's ownership identification. Compared to existing fingerprinting methods, UTF is also more effective and robust to fine-tuning and random guess. |
| title | UTF:Undertrained Tokens as Fingerprints A Novel Approach to LLM Identification |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2410.12318 |