Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911907402743808 |
|---|---|
| author | Huo, Mingjia Somayajula, Sai Ashish Liang, Youwei Zhang, Ruisi Koushanfar, Farinaz Xie, Pengtao |
| author_facet | Huo, Mingjia Somayajula, Sai Ashish Liang, Youwei Zhang, Ruisi Koushanfar, Farinaz Xie, Pengtao |
| contents | Large language models generate high-quality responses with potential misinformation, underscoring the need for regulation by distinguishing AI-generated and human-written texts. Watermarking is pivotal in this context, which involves embedding hidden markers in texts during the LLM inference phase, which is imperceptible to humans. Achieving both the detectability of inserted watermarks and the semantic quality of generated texts is challenging. While current watermarking algorithms have made promising progress in this direction, there remains significant scope for improvement. To address these challenges, we introduce a novel multi-objective optimization (MOO) approach for watermarking that utilizes lightweight networks to generate token-specific watermarking logits and splitting ratios. By leveraging MOO to optimize for both detection and semantic objective functions, our method simultaneously achieves detectability and semantic integrity. Experimental results show that our method outperforms current watermarking techniques in enhancing the detectability of texts generated by LLMs while maintaining their semantic coherence. Our code is available at https://github.com/mignonjia/TS_watermark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_18059 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models Huo, Mingjia Somayajula, Sai Ashish Liang, Youwei Zhang, Ruisi Koushanfar, Farinaz Xie, Pengtao Machine Learning Computation and Language Cryptography and Security Large language models generate high-quality responses with potential misinformation, underscoring the need for regulation by distinguishing AI-generated and human-written texts. Watermarking is pivotal in this context, which involves embedding hidden markers in texts during the LLM inference phase, which is imperceptible to humans. Achieving both the detectability of inserted watermarks and the semantic quality of generated texts is challenging. While current watermarking algorithms have made promising progress in this direction, there remains significant scope for improvement. To address these challenges, we introduce a novel multi-objective optimization (MOO) approach for watermarking that utilizes lightweight networks to generate token-specific watermarking logits and splitting ratios. By leveraging MOO to optimize for both detection and semantic objective functions, our method simultaneously achieves detectability and semantic integrity. Experimental results show that our method outperforms current watermarking techniques in enhancing the detectability of texts generated by LLMs while maintaining their semantic coherence. Our code is available at https://github.com/mignonjia/TS_watermark. |
| title | Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models |
| topic | Machine Learning Computation and Language Cryptography and Security |
| url | https://arxiv.org/abs/2402.18059 |