Cover Image

Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Moshkovich, Dany, Mulian, Hadar, Zeltyn, Sergey, Eder, Natti, Skarbovsky, Inna, Abitbol, Roy
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Multiagent Systems
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems
by: Moshkovich, Dany, et al.
Published: (2025)

AgentFixer: From Failure Detection to Fix Recommendations in LLM Agentic Systems
by: Mulian, Hadar, et al.
Published: (2026)

Beyond the Black Box: Interpretability of Agentic AI Tool Use
by: Tatsat, Hariom, et al.
Published: (2026)

From Grounding to Planning: Benchmarking Bottlenecks in Web Agents
by: Shlomov, Segev, et al.
Published: (2024)

Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization
by: Liu, Shiyan, et al.
Published: (2026)

Interpreting Agentic Systems: Beyond Model Explanations to System-Level Accountability
by: Zhu, Judy, et al.
Published: (2026)

CREW-WILDFIRE: Benchmarking Agentic Multi-Agent Collaborations at Scale
by: Hyun, Jonathan, et al.
Published: (2025)

Agentization of Digital Assets for the Agentic Web: Concepts, Techniques, and Benchmark
by: Chen, Linyao, et al.
Published: (2026)

SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence
by: Zhang, Yao, et al.
Published: (2025)

AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding
by: Chowdhury, Sanjoy, et al.
Published: (2025)

Optimization Instability in Autonomous Agentic Workflows for Clinical Symptom Detection
by: Cagan, Cameron, et al.
Published: (2026)

Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems
by: Akshathala, Sreemaee, et al.
Published: (2025)

Counterfactual-based Agent Influence Ranker for Agentic AI Workflows
by: Giloni, Amit, et al.
Published: (2025)

ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems
by: Zhang, Yifei, et al.
Published: (2026)

SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systems
by: Chillara, Varun, et al.
Published: (2026)

SafeAgent: A Runtime Protection Architecture for Agentic Systems
by: Liu, Hailin, et al.
Published: (2026)

Learning to Share: Selective Memory for Efficient Parallel Agentic Systems
by: Fioresi, Joseph, et al.
Published: (2026)

Robust, Observable, and Evolvable Agentic Systems Engineering: A Principled Framework Validated via the Fairy GUI Agent
by: Sun, Jiazheng, et al.
Published: (2025)

Evolution of Collective AI Beyond Individual Optimization
by: Takata, Ryosuke, et al.
Published: (2024)

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution
by: Zehle, Tom
Published: (2026)

Query-Efficient Agentic Graph Extraction Attacks on GraphRAG Systems
by: Yang, Shuhua, et al.
Published: (2026)

Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading
by: Song, Zifan, et al.
Published: (2025)

Sentinel Agents for Secure and Trustworthy Agentic AI in Multi-Agent Systems
by: Gosmar, Diego, et al.
Published: (2025)

Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations
by: Liu, Bochao, et al.
Published: (2026)

The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems
by: Nakajima, Yohei
Published: (2026)

Revisiting Gossip Protocols: A Vision for Emergent Coordination in Agentic Multi-Agent Systems
by: Habiba, Mansura, et al.
Published: (2025)

Holos: A Web-Scale LLM-Based Multi-Agent System for the Agentic Web
by: Nie, Xiaohang, et al.
Published: (2026)

Agentifying Agentic AI
by: Dignum, Virginia, et al.
Published: (2025)

The Effect of Belief Boxes and Open-mindedness on Persuasion
by: Bilgin, Onur, et al.
Published: (2025)

Intermediate Artifacts as First-Class Citizens: A Data Model for Durable Intermediate Artifacts in Agentic Systems
by: Rosen, Josh, et al.
Published: (2026)

Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems
by: Wang, Haochun, et al.
Published: (2025)

Computing Universal Plans for Partially Observable Multi-Agent Routing Using Answer Set Programming
by: Zhu, Fengming, et al.
Published: (2023)

Aligning Artificial Superintelligence via a Multi-Box Protocol
by: Negozio, Avraham Yair
Published: (2025)

LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers
by: Naug, Avisek, et al.
Published: (2025)

TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems
by: Kavathekar, Ishan, et al.
Published: (2025)

Agentic AI and Multiagentic: Are We Reinventing the Wheel?
by: Botti, V.
Published: (2025)

EnvX: Agentize Everything with Agentic AI
by: Chen, Linyao, et al.
Published: (2025)

Agentic AI Framework for Smart Inventory Replenishment
by: Syed, Toqeer Ali, et al.
Published: (2025)

Agent Identity Evals: Measuring Agentic Identity
by: Perrier, Elija, et al.
Published: (2025)

Agentic Discovery: Closing the Loop with Cooperative Agents
by: Pauloski, J. Gregory, et al.
Published: (2025)