Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909660281307136 |
|---|---|
| author | Sun, Chenkai Zhang, Denghui Zhai, ChengXiang Ji, Heng |
| author_facet | Sun, Chenkai Zhang, Denghui Zhai, ChengXiang Ji, Heng |
| contents | Given the growing influence of language model-based agents on high-stakes societal decisions, from public policy to healthcare, ensuring their beneficial impact requires understanding the far-reaching implications of their suggestions. We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems on a macroscopic scale over time, enabling more robust alignment. To assess the long-term safety awareness of language models, we also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts. Our approach achieves not only over 20% improvement on the new dataset but also an average win rate exceeding 70% against strong baselines on existing safety benchmarks (AdvBench, SafeRLHF, WildGuardMix), suggesting a promising direction for safer agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_20949 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation Sun, Chenkai Zhang, Denghui Zhai, ChengXiang Ji, Heng Artificial Intelligence Computation and Language Given the growing influence of language model-based agents on high-stakes societal decisions, from public policy to healthcare, ensuring their beneficial impact requires understanding the far-reaching implications of their suggestions. We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems on a macroscopic scale over time, enabling more robust alignment. To assess the long-term safety awareness of language models, we also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts. Our approach achieves not only over 20% improvement on the new dataset but also an average win rate exceeding 70% against strong baselines on existing safety benchmarks (AdvBench, SafeRLHF, WildGuardMix), suggesting a promising direction for safer agents. |
| title | Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2506.20949 |