SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912155485339648 |
|---|---|
| author | Banerjee, Somnath Layek, Sayan Tripathy, Soham Kumar, Shanu Mukherjee, Animesh Hazra, Rima |
| author_facet | Banerjee, Somnath Layek, Sayan Tripathy, Soham Kumar, Shanu Mukherjee, Animesh Hazra, Rima |
| contents | Safety-aligned language models often exhibit fragile and imbalanced safety mechanisms, increasing the likelihood of generating unsafe content. In addition, incorporating new knowledge through editing techniques to language models can further compromise safety. To address these issues, we propose SafeInfer, a context-adaptive, decoding-time safety alignment strategy for generating safe responses to user queries. SafeInfer comprises two phases: the safety amplification phase, which employs safe demonstration examples to adjust the model's hidden states and increase the likelihood of safer outputs, and the safety-guided decoding phase, which influences token selection based on safety-optimized distributions, ensuring the generated content complies with ethical guidelines. Further, we present HarmEval, a novel benchmark for extensive safety evaluations, designed to address potential misuse scenarios in accordance with the policies of leading AI tech giants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12274 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models Banerjee, Somnath Layek, Sayan Tripathy, Soham Kumar, Shanu Mukherjee, Animesh Hazra, Rima Computation and Language Safety-aligned language models often exhibit fragile and imbalanced safety mechanisms, increasing the likelihood of generating unsafe content. In addition, incorporating new knowledge through editing techniques to language models can further compromise safety. To address these issues, we propose SafeInfer, a context-adaptive, decoding-time safety alignment strategy for generating safe responses to user queries. SafeInfer comprises two phases: the safety amplification phase, which employs safe demonstration examples to adjust the model's hidden states and increase the likelihood of safer outputs, and the safety-guided decoding phase, which influences token selection based on safety-optimized distributions, ensuring the generated content complies with ethical guidelines. Further, we present HarmEval, a novel benchmark for extensive safety evaluations, designed to address potential misuse scenarios in accordance with the policies of leading AI tech giants. |
| title | SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.12274 |