Exploring Straightforward Conversational Red-Teaming
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917770835263488 |
|---|---|
| author | Kour, George Zwerdling, Naama Zalmanovici, Marcel Anaby-Tavor, Ateret Fandina, Ora Nova Farchi, Eitan |
| author_facet | Kour, George Zwerdling, Naama Zalmanovici, Marcel Anaby-Tavor, Ateret Fandina, Ora Nova Farchi, Eitan |
| contents | Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the model's behavior, can be exploited to produce undesired responses. In this paper, we examine the effectiveness of utilizing off-the-shelf LLMs in straightforward red-teaming approaches, where an attacker LLM aims to elicit undesired output from a target LLM, comparing both single-turn and conversational red-teaming tactics. Our experiments offer insights into various usage strategies that significantly affect their performance as red teamers. They suggest that off-the-shelf models can act as effective red teamers and even adjust their attack strategy based on past attempts, although their effectiveness decreases with greater alignment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_04822 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring Straightforward Conversational Red-Teaming Kour, George Zwerdling, Naama Zalmanovici, Marcel Anaby-Tavor, Ateret Fandina, Ora Nova Farchi, Eitan Computation and Language Artificial Intelligence Large language models (LLMs) are increasingly used in business dialogue systems but they pose security and ethical risks. Multi-turn conversations, where context influences the model's behavior, can be exploited to produce undesired responses. In this paper, we examine the effectiveness of utilizing off-the-shelf LLMs in straightforward red-teaming approaches, where an attacker LLM aims to elicit undesired output from a target LLM, comparing both single-turn and conversational red-teaming tactics. Our experiments offer insights into various usage strategies that significantly affect their performance as red teamers. They suggest that off-the-shelf models can act as effective red teamers and even adjust their attack strategy based on past attempts, although their effectiveness decreases with greater alignment. |
| title | Exploring Straightforward Conversational Red-Teaming |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.04822 |