The 20 questions game to distinguish large language models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917776995647488 |
|---|---|
| author | Richardeau, Gurvan Merrer, Erwan Le Penzo, Camilla Tredan, Gilles |
| author_facet | Richardeau, Gurvan Merrer, Erwan Le Penzo, Camilla Tredan, Gilles |
| contents | In a parallel with the 20 questions game, we present a method to determine whether two large language models (LLMs), placed in a black-box context, are the same or not. The goal is to use a small set of (benign) binary questions, typically under 20. We formalize the problem and first establish a baseline using a random selection of questions from known benchmark datasets, achieving an accuracy of nearly 100% within 20 questions. After showing optimal bounds for this problem, we introduce two effective questioning heuristics able to discriminate 22 LLMs by using half as many questions for the same task. These methods offer significant advantages in terms of stealth and are thus of interest to auditors or copyright owners facing suspicions of model leaks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10338 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The 20 questions game to distinguish large language models Richardeau, Gurvan Merrer, Erwan Le Penzo, Camilla Tredan, Gilles Computation and Language Artificial Intelligence In a parallel with the 20 questions game, we present a method to determine whether two large language models (LLMs), placed in a black-box context, are the same or not. The goal is to use a small set of (benign) binary questions, typically under 20. We formalize the problem and first establish a baseline using a random selection of questions from known benchmark datasets, achieving an accuracy of nearly 100% within 20 questions. After showing optimal bounds for this problem, we introduce two effective questioning heuristics able to discriminate 22 LLMs by using half as many questions for the same task. These methods offer significant advantages in terms of stealth and are thus of interest to auditors or copyright owners facing suspicions of model leaks. |
| title | The 20 questions game to distinguish large language models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.10338 |