MicroRemed: Benchmarking LLMs in Microservices Remediation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909882899234816 |
|---|---|
| author | Zhang, Lingzhe Zhai, Yunpeng Jia, Tong Duan, Chiming He, Minghua Pan, Leyi Liu, Zhaoyang Ding, Bolin Li, Ying |
| author_facet | Zhang, Lingzhe Zhai, Yunpeng Jia, Tong Duan, Chiming He, Minghua Pan, Leyi Liu, Zhaoyang Ding, Bolin Li, Ying |
| contents | Large Language Models (LLMs) integrated with agent-based reasoning frameworks have recently shown strong potential for autonomous decision-making and system-level operations. One promising yet underexplored direction is microservice remediation, where the goal is to automatically recover faulty microservice systems. Existing approaches, however, still rely on human-crafted prompts from Site Reliability Engineers (SREs), with LLMs merely converting textual instructions into executable code. To advance research in this area, we introduce MicroRemed, the first benchmark for evaluating LLMs in end-to-end microservice remediation, where models must directly generate executable Ansible playbooks from diagnosis reports to restore system functionality. We further propose ThinkRemed, a multi-agent framework that emulates the reflective and perceptive reasoning of SREs. Experimental results show that MicroRemed presents substantial challenges to current LLMs, while ThinkRemed improves end-to-end remediation performance through iterative reasoning and system reflection. The benchmark is available at https://github.com/LLM4AIOps/MicroRemed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_01166 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MicroRemed: Benchmarking LLMs in Microservices Remediation Zhang, Lingzhe Zhai, Yunpeng Jia, Tong Duan, Chiming He, Minghua Pan, Leyi Liu, Zhaoyang Ding, Bolin Li, Ying Computation and Language Software Engineering 68T50 I.2.7 Large Language Models (LLMs) integrated with agent-based reasoning frameworks have recently shown strong potential for autonomous decision-making and system-level operations. One promising yet underexplored direction is microservice remediation, where the goal is to automatically recover faulty microservice systems. Existing approaches, however, still rely on human-crafted prompts from Site Reliability Engineers (SREs), with LLMs merely converting textual instructions into executable code. To advance research in this area, we introduce MicroRemed, the first benchmark for evaluating LLMs in end-to-end microservice remediation, where models must directly generate executable Ansible playbooks from diagnosis reports to restore system functionality. We further propose ThinkRemed, a multi-agent framework that emulates the reflective and perceptive reasoning of SREs. Experimental results show that MicroRemed presents substantial challenges to current LLMs, while ThinkRemed improves end-to-end remediation performance through iterative reasoning and system reflection. The benchmark is available at https://github.com/LLM4AIOps/MicroRemed. |
| title | MicroRemed: Benchmarking LLMs in Microservices Remediation |
| topic | Computation and Language Software Engineering 68T50 I.2.7 |
| url | https://arxiv.org/abs/2511.01166 |