An Empirical Study of Bugs in Modern LLM Agent Frameworks
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912930773073920 |
|---|---|
| author | Zhu, Xinxue Wu, Jiacong Zhang, Xiaoyu Li, Tianlin Mu, Yanzhou Zhai, Juan Shen, Chao Fang, Chunrong Liu, Yang |
| author_facet | Zhu, Xinxue Wu, Jiacong Zhang, Xiaoyu Li, Tianlin Mu, Yanzhou Zhai, Juan Shen, Chao Fang, Chunrong Liu, Yang |
| contents | LLM agents have been widely adopted in real-world applications, relying on agent frameworks for workflow execution and multi-agent coordination. As these systems scale, understanding bugs in the underlying agent frameworks becomes critical. However, existing work mainly focuses on agent-level failures, overlooking framework-level bugs. To address this gap, we conduct an empirical study of 998 bug reports from CrewAI and LangChain, constructing a taxonomy of 15 root causes and 7 observable symptoms across five agent lifecycle stages: 'Agent Initialization','Perception', 'Self-Action', 'Mutual Interaction' and 'Evolution'. Our findings show that agent framework bugs mainly arise from 'API misuse', 'API incompatibility', and 'Documentation Desync', largely concentrated in the 'Self-Action' stage. Symptoms typically appear as 'Functional Error', 'Crash', and 'Build Failure', reflecting disruptions to task progression and control flow. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_21806 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | An Empirical Study of Bugs in Modern LLM Agent Frameworks Zhu, Xinxue Wu, Jiacong Zhang, Xiaoyu Li, Tianlin Mu, Yanzhou Zhai, Juan Shen, Chao Fang, Chunrong Liu, Yang Software Engineering LLM agents have been widely adopted in real-world applications, relying on agent frameworks for workflow execution and multi-agent coordination. As these systems scale, understanding bugs in the underlying agent frameworks becomes critical. However, existing work mainly focuses on agent-level failures, overlooking framework-level bugs. To address this gap, we conduct an empirical study of 998 bug reports from CrewAI and LangChain, constructing a taxonomy of 15 root causes and 7 observable symptoms across five agent lifecycle stages: 'Agent Initialization','Perception', 'Self-Action', 'Mutual Interaction' and 'Evolution'. Our findings show that agent framework bugs mainly arise from 'API misuse', 'API incompatibility', and 'Documentation Desync', largely concentrated in the 'Self-Action' stage. Symptoms typically appear as 'Functional Error', 'Crash', and 'Build Failure', reflecting disruptions to task progression and control flow. |
| title | An Empirical Study of Bugs in Modern LLM Agent Frameworks |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2602.21806 |