Advancing LLM Safe Alignment with Safety Representation Ranking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910959924150272 |
|---|---|
| author | Du, Tianqi Wei, Zeming Chen, Quan Zhang, Chenheng Wang, Yisen |
| author_facet | Du, Tianqi Wei, Zeming Chen, Quan Zhang, Chenheng Wang, Yisen |
| contents | The rapid advancement of large language models (LLMs) has demonstrated milestone success in a variety of tasks, yet their potential for generating harmful content has raised significant safety concerns. Existing safety evaluation approaches typically operate directly on textual responses, overlooking the rich information embedded in the model's internal representations. In this paper, we propose Safety Representation Ranking (SRR), a listwise ranking framework that selects safe responses using hidden states from the LLM itself. SRR encodes both instructions and candidate completions using intermediate transformer representations and ranks candidates via a lightweight similarity-based scorer. Our approach directly leverages internal model states and supervision at the list level to capture subtle safety signals. Experiments across multiple benchmarks show that SRR significantly improves robustness to adversarial prompts. Our code will be available upon publication. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_15710 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Advancing LLM Safe Alignment with Safety Representation Ranking Du, Tianqi Wei, Zeming Chen, Quan Zhang, Chenheng Wang, Yisen Computation and Language Machine Learning The rapid advancement of large language models (LLMs) has demonstrated milestone success in a variety of tasks, yet their potential for generating harmful content has raised significant safety concerns. Existing safety evaluation approaches typically operate directly on textual responses, overlooking the rich information embedded in the model's internal representations. In this paper, we propose Safety Representation Ranking (SRR), a listwise ranking framework that selects safe responses using hidden states from the LLM itself. SRR encodes both instructions and candidate completions using intermediate transformer representations and ranks candidates via a lightweight similarity-based scorer. Our approach directly leverages internal model states and supervision at the list level to capture subtle safety signals. Experiments across multiple benchmarks show that SRR significantly improves robustness to adversarial prompts. Our code will be available upon publication. |
| title | Advancing LLM Safe Alignment with Safety Representation Ranking |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2505.15710 |