Cover Image

Continuous Benchmark Generation for Evaluating Enterprise-scale LLM Agents

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Saxena, Divyanshu, Maurya, Rishikesh, Ou, Xiaoxuan, Somashekar, Gagan, Gupta, Shachee Mishra, Iyer, Arun, Kang, Yu, Bansal, Chetan, Akella, Aditya, Rajmohan, Saravan
Format:	Preprint
Published:	2025
Subjects:	Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Building AI Agents for Autonomous Clouds: Challenges and Design Principles
by: Shetty, Manish, et al.
Published: (2024)

AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds
by: Chen, Yinfang, et al.
Published: (2025)

COSMIR: Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context
by: Gupta, Naman, et al.
Published: (2025)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
by: Gupta, Taneesh, et al.
Published: (2025)

REFA: Reference Free Alignment for multi-preference optimization
by: Gupta, Taneesh, et al.
Published: (2024)

Vulcan: Instance-Optimal Systems Heuristics Through LLM-Driven Search
by: Dwivedula, Rohit, et al.
Published: (2025)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents
by: Gupta, Naman, et al.
Published: (2026)

A Joint Learning Approach to Hardware Caching and Prefetching
by: Yuan, Samuel, et al.
Published: (2025)

Man-Made Heuristics Are Dead. Long Live Code Generators!
by: Dwivedula, Rohit, et al.
Published: (2025)

CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents
by: Fu, Wenjie, et al.
Published: (2026)

Exploring LLM-based Agents for Root Cause Analysis
by: Roy, Devjeet, et al.
Published: (2024)

Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4
by: Zhang, Xuchao, et al.
Published: (2024)

Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach
by: Srinivas, Pooja, et al.
Published: (2024)

AutoAdapt: An Automated Domain Adaptation Framework for LLMs
by: Sinha, Sidharth, et al.
Published: (2026)

X-lifecycle Learning for Cloud Incident Management using LLMs
by: Goel, Drishti, et al.
Published: (2024)

Canopy: Property-Driven Learning for Congestion Control
by: Yang, Chenxi, et al.
Published: (2024)

CONGO: Compressive Online Gradient Optimization
by: Carleton, Jeremy, et al.
Published: (2024)

CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling
by: Gupta, Taneesh, et al.
Published: (2024)

Attention Enhanced Entity Recommendation for Intelligent Monitoring in Cloud Systems
by: Hussain, Fiza, et al.
Published: (2025)

Dependency Aware Incident Linking in Large Cloud Systems
by: Ghosh, Supriyo, et al.
Published: (2024)

EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair
by: Zhao, Chenyu, et al.
Published: (2026)

eARCO: Efficient Automated Root Cause Analysis with Prompt Optimization
by: Goel, Drishti, et al.
Published: (2025)

Minerva: A Programmable Memory Test Benchmark for Language Models
by: Xia, Menglin, et al.
Published: (2025)

Contrastive Attribution in the Wild: An Interpretability Analysis of LLM Failures on Realistic Benchmarks
by: Tan, Rongyuan, et al.
Published: (2026)

On the Fundamental Limitations of Decentralized Learnable Reward Shaping in Cooperative Multi-Agent Reinforcement Learning
by: Akella, Aditya
Published: (2025)

Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection
by: Liu, Jun, et al.
Published: (2024)

Memora: A Harmonic Memory Representation Balancing Abstraction and Specificity
by: Xia, Menglin, et al.
Published: (2026)

Intent-based System Design and Operation
by: Anand, Vaastav, et al.
Published: (2025)

Cost-Aware Retrieval-Augmentation Reasoning Models with Adaptive Retrieval Depth
by: Hashemi, Helia, et al.
Published: (2025)

Design and Implementation of a Multiuser Enterprise Resource Planning Solution for Higher Education Institutions: Enhancing Accreditation and Administration
by: Divyansh Bansal, et al.
Published: (2026)

Navigating Rifts in Human-LLM Grounding: Study and Benchmark
by: Shaikh, Omar, et al.
Published: (2025)

SageServe: Optimizing LLM Serving on Cloud Data Centers with Forecast Aware Auto-Scaling
by: Jaiswal, Shashwat, et al.
Published: (2025)

Synergistic Weak-Strong Collaboration by Aligning Preferences
by: Jiao, Yizhu, et al.
Published: (2025)

Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
by: Zhao, Chenyu, et al.
Published: (2026)

DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
by: Ma, Ming, et al.
Published: (2025)

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
by: Jain, Kunal, et al.
Published: (2024)

Zero Data Retention in LLM-based Enterprise AI Assistants: A Comparative Study of Market Leading Agentic AI Products
by: Gupta, Komal, et al.
Published: (2025)

A Benchmark for Language Models in Real-World System Building
by: Jin, Weilin, et al.
Published: (2026)

Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics
by: Akella, Aditya, et al.
Published: (2025)