Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914384094167040 |
|---|---|
| author | Lukáš, Ondřej Shin, Jihoon Rivas, Emilia Forni, Diego Rigaki, Maria Catania, Carlos Piplai, Aritran Kiekintveld, Christopher Garcia, Sebastian |
| author_facet | Lukáš, Ondřej Shin, Jihoon Rivas, Emilia Forni, Diego Rigaki, Maria Catania, Carlos Piplai, Aritran Kiekintveld, Christopher Garcia, Sebastian |
| contents | Autonomous offensive agents often fail to transfer beyond the networks on which they are trained. We isolate a minimal but fundamental shift -- unseen host/subnet IP reassignment in an otherwise fixed enterprise scenario -- and evaluate attacker generalization in the NetSecGame environment. Agents are trained on five IP-range variants and tested on a sixth unseen variant; only the meta-learning agent may adapt at test time. We compare three agent families (traditional RL, adaptation agents, and LLM-based agents) and use action-distribution-based behavioral/XAI analyses to localize failure modes. Some adaptation methods show partial transfer but significant degradation under unseen reassignment, indicating that even address-space changes can break long-horizon attack policies. Under our evaluation protocol and agent-specific assumptions, prompt-driven pretrained LLM agents achieve the highest success on the held-out reassignment, but at the cost of increased inference-time compute, reduced transparency, and practical failure modes such as repetition/invalid-action loops. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_10041 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents Lukáš, Ondřej Shin, Jihoon Rivas, Emilia Forni, Diego Rigaki, Maria Catania, Carlos Piplai, Aritran Kiekintveld, Christopher Garcia, Sebastian Cryptography and Security Machine Learning Autonomous offensive agents often fail to transfer beyond the networks on which they are trained. We isolate a minimal but fundamental shift -- unseen host/subnet IP reassignment in an otherwise fixed enterprise scenario -- and evaluate attacker generalization in the NetSecGame environment. Agents are trained on five IP-range variants and tested on a sixth unseen variant; only the meta-learning agent may adapt at test time. We compare three agent families (traditional RL, adaptation agents, and LLM-based agents) and use action-distribution-based behavioral/XAI analyses to localize failure modes. Some adaptation methods show partial transfer but significant degradation under unseen reassignment, indicating that even address-space changes can break long-horizon attack policies. Under our evaluation protocol and agent-specific assumptions, prompt-driven pretrained LLM agents achieve the highest success on the held-out reassignment, but at the cost of increased inference-time compute, reduced transparency, and practical failure modes such as repetition/invalid-action loops. |
| title | Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2603.10041 |