The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917266238472192 |
|---|---|
| author | Wang, Chenxu Li, Chaozhuo Liu, Songyang Chen, Zejian Hou, Jinyu Qi, Ji Li, Rui Zhang, Litian Ye, Qiwei Liu, Zheng Chen, Xu Zhang, Xi Yu, Philip S. |
| author_facet | Wang, Chenxu Li, Chaozhuo Liu, Songyang Chen, Zejian Hou, Jinyu Qi, Ji Li, Rui Zhang, Litian Ye, Qiwei Liu, Zheng Chen, Xu Zhang, Xi Yu, Philip S. |
| contents | The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_09877 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies Wang, Chenxu Li, Chaozhuo Liu, Songyang Chen, Zejian Hou, Jinyu Qi, Ji Li, Rui Zhang, Litian Ye, Qiwei Liu, Zheng Chen, Xu Zhang, Xi Yu, Philip S. Computation and Language The emergence of multi-agent systems built from large language models (LLMs) offers a promising paradigm for scalable collective intelligence and self-evolution. Ideally, such systems would achieve continuous self-improvement in a fully closed loop while maintaining robust safety alignment--a combination we term the self-evolution trilemma. However, we demonstrate both theoretically and empirically that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible. Drawing on an information-theoretic framework, we formalize safety as the divergence degree from anthropic value distributions. We theoretically demonstrate that isolated self-evolution induces statistical blind spots, leading to the irreversible degradation of the system's safety alignment. Empirical and qualitative results from an open-ended agent community (Moltbook) and two closed self-evolving systems reveal phenomena that align with our theoretical prediction of inevitable safety erosion. We further propose several solution directions to alleviate the identified safety concern. Our work establishes a fundamental limit on the self-evolving AI societies and shifts the discourse from symptom-driven safety patches to a principled understanding of intrinsic dynamical risks, highlighting the need for external oversight or novel safety-preserving mechanisms. |
| title | The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.09877 |