AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912619796889600 |
|---|---|
| author | Naik, Akshat Quinn, Patrick Bosch, Guillermo Gouné, Emma Zabala, Francisco Javier Campos Brown, Jason Ross Young, Edward James |
| author_facet | Naik, Akshat Quinn, Patrick Bosch, Guillermo Gouné, Emma Zabala, Francisco Javier Campos Brown, Jason Ross Young, Edward James |
| contents | As Large Language Model (LLM) agents become more widespread, associated misalignment risks increase. While prior research has studied agents' ability to produce harmful outputs or follow malicious instructions, it remains unclear how likely agents are to spontaneously pursue unintended goals in realistic deployments. In this work, we approach misalignment as a conflict between the internal goals pursued by the model and the goals intended by its deployer. We introduce a misalignment propensity benchmark, \textsc{AgentMisalignment}, a benchmark suite designed to evaluate the propensity of LLM agents to misalign in realistic scenarios. Evaluations cover behaviours such as avoiding oversight, resisting shutdown, sandbagging, and power-seeking. Testing frontier models, we find that more capable agents tend to exhibit higher misalignment on average. We also systematically vary agent personalities through different system prompts and observe that persona characteristics can strongly and unpredictably influence misalignment, sometimes more than the choice of model itself. Our results reveal the limitations of current alignment methods for autonomous LLM agents and underscore the need to rethink misalignment in realistic deployment settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04018 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents Naik, Akshat Quinn, Patrick Bosch, Guillermo Gouné, Emma Zabala, Francisco Javier Campos Brown, Jason Ross Young, Edward James Artificial Intelligence Computation and Language Computers and Society Machine Learning I.2.7; I.2.11; K.4.1; I.2.6 As Large Language Model (LLM) agents become more widespread, associated misalignment risks increase. While prior research has studied agents' ability to produce harmful outputs or follow malicious instructions, it remains unclear how likely agents are to spontaneously pursue unintended goals in realistic deployments. In this work, we approach misalignment as a conflict between the internal goals pursued by the model and the goals intended by its deployer. We introduce a misalignment propensity benchmark, \textsc{AgentMisalignment}, a benchmark suite designed to evaluate the propensity of LLM agents to misalign in realistic scenarios. Evaluations cover behaviours such as avoiding oversight, resisting shutdown, sandbagging, and power-seeking. Testing frontier models, we find that more capable agents tend to exhibit higher misalignment on average. We also systematically vary agent personalities through different system prompts and observe that persona characteristics can strongly and unpredictably influence misalignment, sometimes more than the choice of model itself. Our results reveal the limitations of current alignment methods for autonomous LLM agents and underscore the need to rethink misalignment in realistic deployment settings. |
| title | AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents |
| topic | Artificial Intelligence Computation and Language Computers and Society Machine Learning I.2.7; I.2.11; K.4.1; I.2.6 |
| url | https://arxiv.org/abs/2506.04018 |