Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913603883368448 |
|---|---|
| author | Fernandez, Ivan A. Neupane, Subash Mittal, Sudip Rahimi, Shahram |
| author_facet | Fernandez, Ivan A. Neupane, Subash Mittal, Sudip Rahimi, Shahram |
| contents | Recent research has shown that large language models (LLMs) are particularly vulnerable to adversarial attacks. Since the release of ChatGPT, various industries are adopting LLM-based chatbots and virtual assistants in their data workflows. The rapid development pace of AI-based systems is being driven by the potential of Generative AI (GenAI) to assist humans in decision making. The immense optimism behind GenAI often overshadows the adversarial risks associated with these technologies. A threat actor can use security gaps, poor safeguards, and limited data governance to carry out attacks that grant unauthorized access to the system and its data. As a proof-of-concept, we assess the performance of BarkPlug, the Mississippi State University chatbot, against data poison attacks from a red team perspective. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_06788 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant Fernandez, Ivan A. Neupane, Subash Mittal, Sudip Rahimi, Shahram Cryptography and Security Recent research has shown that large language models (LLMs) are particularly vulnerable to adversarial attacks. Since the release of ChatGPT, various industries are adopting LLM-based chatbots and virtual assistants in their data workflows. The rapid development pace of AI-based systems is being driven by the potential of Generative AI (GenAI) to assist humans in decision making. The immense optimism behind GenAI often overshadows the adversarial risks associated with these technologies. A threat actor can use security gaps, poor safeguards, and limited data governance to carry out attacks that grant unauthorized access to the system and its data. As a proof-of-concept, we assess the performance of BarkPlug, the Mississippi State University chatbot, against data poison attacks from a red team perspective. |
| title | Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant |
| topic | Cryptography and Security |
| url | https://arxiv.org/abs/2412.06788 |