Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.14195 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912810116579328 |
|---|---|
| author | Lin, Zheyu Yang, Jirui Qiu, Yukui Guo, Hengqi Bao, Yubing Guan, Yao |
| author_facet | Lin, Zheyu Yang, Jirui Qiu, Yukui Guo, Hengqi Bao, Yubing Guan, Yao |
| contents | Evaluating the safety robustness of LLMs is critical for their deployment. However, mainstream Red Teaming methods rely on online generation and black-box output analysis. These approaches are not only costly but also suffer from feedback latency, making them unsuitable for agile diagnostics after training a new model. To address this, we propose N-GLARE (A Non-Generative, Latent Representation-Efficient LLM Safety Evaluator). N-GLARE operates entirely on the model's latent representations, bypassing the need for full text generation. It characterizes hidden layer dynamics by analyzing the APT (Angular-Probabilistic Trajectory) of latent representations and introducing the JSS (Jensen-Shannon Separability) metric. Experiments on over 40 models and 20 red teaming strategies demonstrate that the JSS metric exhibits high consistency with the safety rankings derived from Red Teaming. N-GLARE reproduces the discriminative trends of large-scale red-teaming tests at less than 1\% of the token cost and the runtime cost, providing an efficient output-free evaluation proxy for real-time diagnostics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_14195 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluator Lin, Zheyu Yang, Jirui Qiu, Yukui Guo, Hengqi Bao, Yubing Guan, Yao Machine Learning Cryptography and Security Evaluating the safety robustness of LLMs is critical for their deployment. However, mainstream Red Teaming methods rely on online generation and black-box output analysis. These approaches are not only costly but also suffer from feedback latency, making them unsuitable for agile diagnostics after training a new model. To address this, we propose N-GLARE (A Non-Generative, Latent Representation-Efficient LLM Safety Evaluator). N-GLARE operates entirely on the model's latent representations, bypassing the need for full text generation. It characterizes hidden layer dynamics by analyzing the APT (Angular-Probabilistic Trajectory) of latent representations and introducing the JSS (Jensen-Shannon Separability) metric. Experiments on over 40 models and 20 red teaming strategies demonstrate that the JSS metric exhibits high consistency with the safety rankings derived from Red Teaming. N-GLARE reproduces the discriminative trends of large-scale red-teaming tests at less than 1\% of the token cost and the runtime cost, providing an efficient output-free evaluation proxy for real-time diagnostics. |
| title | N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluator |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2511.14195 |