Towards Provably Secure Generative AI: Reliable Consensus Sampling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909979117617152 |
|---|---|
| author | Cui, Yu Fu, Hang Pan, Sicheng Sun, Zhuoyu Liu, Yifei Nie, Yuhong Ran, Bo Huang, Baohan Zhang, Xufeng Zhang, Haibin Zuo, Cong Wang, Licheng |
| author_facet | Cui, Yu Fu, Hang Pan, Sicheng Sun, Zhuoyu Liu, Yifei Nie, Yuhong Ran, Bo Huang, Baohan Zhang, Xufeng Zhang, Haibin Zuo, Cong Wang, Licheng |
| contents | Existing research on generative AI security is primarily driven by mutually reinforcing attack and defense methodologies grounded in empirical experience. This dynamic frequently gives rise to previously unknown attacks that can circumvent current detection and prevention. This necessitates the continual updating of security mechanisms. Constructing generative AI with provable security and theoretically controllable risk is therefore necessary. Consensus Sampling (CS) is a promising algorithm toward provably secure AI. It controls risk by leveraging overlap in model output probabilities. However, we find that CS relies on frequent abstention to avoid unsafe outputs, which reduces utility. Moreover, CS becomes highly vulnerable when unsafe models are maliciously manipulated. To address these issues, we propose a new primitive called Reliable Consensus Sampling (RCS), that traces acceptance probability to tolerate extreme adversarial behaviors, improving robustness. RCS also eliminates the need for abstention entirely. We further develop a feedback algorithm to continuously and dynamically enhance the safety of RCS. We provide theoretical guarantees that RCS maintains a controllable risk threshold. Extensive experiments show that RCS significantly improves robustness and utility while maintaining latency comparable to CS. We hope this work contributes to the development of provably secure generative AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_24925 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Provably Secure Generative AI: Reliable Consensus Sampling Cui, Yu Fu, Hang Pan, Sicheng Sun, Zhuoyu Liu, Yifei Nie, Yuhong Ran, Bo Huang, Baohan Zhang, Xufeng Zhang, Haibin Zuo, Cong Wang, Licheng Cryptography and Security Existing research on generative AI security is primarily driven by mutually reinforcing attack and defense methodologies grounded in empirical experience. This dynamic frequently gives rise to previously unknown attacks that can circumvent current detection and prevention. This necessitates the continual updating of security mechanisms. Constructing generative AI with provable security and theoretically controllable risk is therefore necessary. Consensus Sampling (CS) is a promising algorithm toward provably secure AI. It controls risk by leveraging overlap in model output probabilities. However, we find that CS relies on frequent abstention to avoid unsafe outputs, which reduces utility. Moreover, CS becomes highly vulnerable when unsafe models are maliciously manipulated. To address these issues, we propose a new primitive called Reliable Consensus Sampling (RCS), that traces acceptance probability to tolerate extreme adversarial behaviors, improving robustness. RCS also eliminates the need for abstention entirely. We further develop a feedback algorithm to continuously and dynamically enhance the safety of RCS. We provide theoretical guarantees that RCS maintains a controllable risk threshold. Extensive experiments show that RCS significantly improves robustness and utility while maintaining latency comparable to CS. We hope this work contributes to the development of provably secure generative AI. |
| title | Towards Provably Secure Generative AI: Reliable Consensus Sampling |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2512.24925 |