Lessons from Defending Gemini Against Indirect Prompt Injections
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909617537155072 |
|---|---|
| author | Shi, Chongyang Lin, Sharon Song, Shuang Hayes, Jamie Shumailov, Ilia Yona, Itay Pluto, Juliette Pappu, Aneesh Choquette-Choo, Christopher A. Nasr, Milad Sitawarin, Chawin Gibson, Gena Terzis, Andreas Flynn, John "Four" |
| author_facet | Shi, Chongyang Lin, Sharon Song, Shuang Hayes, Jamie Shumailov, Ilia Yona, Itay Pluto, Juliette Pappu, Aneesh Choquette-Choo, Christopher A. Nasr, Milad Sitawarin, Chawin Gibson, Gena Terzis, Andreas Flynn, John "Four" |
| contents | Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require access to untrusted data introducing risk. Adversaries can embed malicious instructions in untrusted data which cause the model to deviate from the user's expectations and mishandle their data or permissions. In this report, we set out Google DeepMind's approach to evaluating the adversarial robustness of Gemini models and describe the main lessons learned from the process. We test how Gemini performs against a sophisticated adversary through an adversarial evaluation framework, which deploys a suite of adaptive attack techniques to run continuously against past, current, and future versions of Gemini. We describe how these ongoing evaluations directly help make Gemini more resilient against manipulation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14534 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Lessons from Defending Gemini Against Indirect Prompt Injections Shi, Chongyang Lin, Sharon Song, Shuang Hayes, Jamie Shumailov, Ilia Yona, Itay Pluto, Juliette Pappu, Aneesh Choquette-Choo, Christopher A. Nasr, Milad Sitawarin, Chawin Gibson, Gena Terzis, Andreas Flynn, John "Four" Cryptography and Security Machine Learning Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require access to untrusted data introducing risk. Adversaries can embed malicious instructions in untrusted data which cause the model to deviate from the user's expectations and mishandle their data or permissions. In this report, we set out Google DeepMind's approach to evaluating the adversarial robustness of Gemini models and describe the main lessons learned from the process. We test how Gemini performs against a sophisticated adversary through an adversarial evaluation framework, which deploys a suite of adaptive attack techniques to run continuously against past, current, and future versions of Gemini. We describe how these ongoing evaluations directly help make Gemini more resilient against manipulation. |
| title | Lessons from Defending Gemini Against Indirect Prompt Injections |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2505.14534 |