Toward Architecture-Aware Evaluation Metrics for LLM Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911402558488576 |
|---|---|
| author | Souza, Débora Machado, Patrícia |
| author_facet | Souza, Débora Machado, Patrícia |
| contents | LLM-based agents are becoming central to software engineering tasks, yet evaluating them remains fragmented and largely model-centric. Existing studies overlook how architectural components, such as planners, memory, and tool routers, shape agent behavior, limiting diagnostic power. We propose a lightweight, architecture-informed approach that links agent components to their observable behaviors and to the metrics capable of evaluating them. Our method clarifies what to measure and why, and we illustrate its application through real world agents, enabling more targeted, transparent, and actionable evaluation of LLM-based agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_19583 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Toward Architecture-Aware Evaluation Metrics for LLM Agents Souza, Débora Machado, Patrícia Software Engineering D.2.7; I.2.2; I.2.7 LLM-based agents are becoming central to software engineering tasks, yet evaluating them remains fragmented and largely model-centric. Existing studies overlook how architectural components, such as planners, memory, and tool routers, shape agent behavior, limiting diagnostic power. We propose a lightweight, architecture-informed approach that links agent components to their observable behaviors and to the metrics capable of evaluating them. Our method clarifies what to measure and why, and we illustrate its application through real world agents, enabling more targeted, transparent, and actionable evaluation of LLM-based agents. |
| title | Toward Architecture-Aware Evaluation Metrics for LLM Agents |
| topic | Software Engineering D.2.7; I.2.2; I.2.7 |
| url | https://arxiv.org/abs/2601.19583 |