Debiasing Watermarks for Large Language Models via Maximal Coupling
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908405366521856 |
|---|---|
| author | Xie, Yangxinyu Li, Xiang Mallick, Tanwi Su, Weijie J. Zhang, Ruixun |
| author_facet | Xie, Yangxinyu Li, Xiang Mallick, Tanwi Su, Weijie J. Zhang, Ruixun |
| contents | Watermarking language models is essential for distinguishing between human and machine-generated text and thus maintaining the integrity and trustworthiness of digital communication. We present a novel green/red list watermarking approach that partitions the token set into ``green'' and ``red'' lists, subtly increasing the generation probability for green tokens. To correct token distribution bias, our method employs maximal coupling, using a uniform coin flip to decide whether to apply bias correction, with the result embedded as a pseudorandom watermark signal. Theoretical analysis confirms this approach's unbiased nature and robust detection capabilities. Experimental results show that it outperforms prior techniques by preserving text quality while maintaining high detectability, and it demonstrates resilience to targeted modifications aimed at improving text quality. This research provides a promising watermarking solution for language models, balancing effective detection with minimal impact on text quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11203 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Debiasing Watermarks for Large Language Models via Maximal Coupling Xie, Yangxinyu Li, Xiang Mallick, Tanwi Su, Weijie J. Zhang, Ruixun Machine Learning Computation and Language Cryptography and Security Methodology Watermarking language models is essential for distinguishing between human and machine-generated text and thus maintaining the integrity and trustworthiness of digital communication. We present a novel green/red list watermarking approach that partitions the token set into ``green'' and ``red'' lists, subtly increasing the generation probability for green tokens. To correct token distribution bias, our method employs maximal coupling, using a uniform coin flip to decide whether to apply bias correction, with the result embedded as a pseudorandom watermark signal. Theoretical analysis confirms this approach's unbiased nature and robust detection capabilities. Experimental results show that it outperforms prior techniques by preserving text quality while maintaining high detectability, and it demonstrates resilience to targeted modifications aimed at improving text quality. This research provides a promising watermarking solution for language models, balancing effective detection with minimal impact on text quality. |
| title | Debiasing Watermarks for Large Language Models via Maximal Coupling |
| topic | Machine Learning Computation and Language Cryptography and Security Methodology |
| url | https://arxiv.org/abs/2411.11203 |