Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917898150215680 |
|---|---|
| author | Keane, Jonathan Keyser, Sam Kedziora, Jeremy |
| author_facet | Keane, Jonathan Keyser, Sam Kedziora, Jeremy |
| contents | The use of reward functions to structure AI learning and decision making is core to the current reinforcement learning paradigm; however, without careful design of reward functions, agents can learn to solve problems in ways that may be considered "undesirable" or "unethical." Without thorough understanding of the incentives a reward function creates, it can be difficult to impose principled yet general control mechanisms over its behavior. In this paper, we study methods for constructing guardrails for AI agents that use reward functions to learn decision making. We introduce a novel approach, which we call strategy masking, to explicitly learn and then suppress undesirable AI agent behavior. We apply our method to study lying in AI agents and show that it can be used to effectively modify agent behavior by suppressing lying post-training without compromising agent ability to perform effectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_05501 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents Keane, Jonathan Keyser, Sam Kedziora, Jeremy Artificial Intelligence Machine Learning Multiagent Systems I.2.0 The use of reward functions to structure AI learning and decision making is core to the current reinforcement learning paradigm; however, without careful design of reward functions, agents can learn to solve problems in ways that may be considered "undesirable" or "unethical." Without thorough understanding of the incentives a reward function creates, it can be difficult to impose principled yet general control mechanisms over its behavior. In this paper, we study methods for constructing guardrails for AI agents that use reward functions to learn decision making. We introduce a novel approach, which we call strategy masking, to explicitly learn and then suppress undesirable AI agent behavior. We apply our method to study lying in AI agents and show that it can be used to effectively modify agent behavior by suppressing lying post-training without compromising agent ability to perform effectively. |
| title | Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents |
| topic | Artificial Intelligence Machine Learning Multiagent Systems I.2.0 |
| url | https://arxiv.org/abs/2501.05501 |