Cover Image

Measuring Agents in Production

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pan, Melissa Z., Arabzadeh, Negar, Cogo, Riccardo, Zhu, Yuxuan, Xiong, Alexander, Agrawal, Lakshya A, Mao, Huanzhi, Shen, Emma, Pallerla, Sid, Patel, Liana, Liu, Shu, Shi, Tianneng, Liu, Xiaoyuan, Davis, Jared Quincy, Lacavalla, Emmanuele, Basile, Alessandro, Yang, Shuyi, Castro, Paul, Kang, Daniel, Gonzalez, Joseph E., Sen, Koushik, Song, Dawn, Stoica, Ion, Zaharia, Matei, Ellis, Marquita
Format:	Preprint
Published:	2025
Subjects:	Computers and Society Artificial Intelligence Machine Learning Software Engineering
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis
by: Patel, Liana, et al.
Published: (2025)

Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design
by: Davis, Jared Quincy, et al.
Published: (2024)

Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines
by: Arabzadeh, Negar, et al.
Published: (2026)

RAG over Thinking Traces Can Improve Reasoning Tasks
by: Arabzadeh, Negar, et al.
Published: (2026)

Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems
by: Chen, Lingjiao, et al.
Published: (2024)

Optimizing Model Selection for Compound AI Systems
by: Chen, Lingjiao, et al.
Published: (2025)

BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
by: Zhu, Alan, et al.
Published: (2025)

HEnRY: A Multi-Agent System Framework for Multi-Domain Contexts
by: Lacavalla, Emmanuele, et al.
Published: (2024)

Natural Language Query to Configuration for Retrieval Agents
by: Pan, Melissa Z., et al.
Published: (2026)

The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More
by: Chen, Lingjiao, et al.
Published: (2026)

HashAttention: Semantic Sparsity for Faster Inference
by: Desai, Aditya, et al.
Published: (2024)

DeServe: Towards Affordable Offline LLM Inference via Decentralization
by: Wu, Linyu, et al.
Published: (2025)

ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data
by: Patel, Liana, et al.
Published: (2024)

Specifications: The missing link to making the development of LLM systems an engineering discipline
by: Stoica, Ion, et al.
Published: (2024)

Why Do Multi-Agent LLM Systems Fail?
by: Cemri, Mert, et al.
Published: (2025)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
by: Cao, Shiyi, et al.
Published: (2024)

LLM CHESS: Benchmarking Reasoning and Instruction-Following in LLMs through Chess
by: Kolasani, Sai, et al.
Published: (2025)

RAFT: Adapting Language Model to Domain Specific RAG
by: Zhang, Tianjun, et al.
Published: (2024)

AI-Driven Research for Databases
by: Cheng, Audrey, et al.
Published: (2026)

Optimizing LLM Queries in Relational Data Analytics Workloads
by: Liu, Shu, et al.
Published: (2024)

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
by: Cemri, Mert, et al.
Published: (2026)

Barbarians at the Gate: How AI is Upending Systems Research
by: Cheng, Audrey, et al.
Published: (2025)

optimize_anything: A Universal API for Optimizing any Text Parameter
by: Agrawal, Lakshya A, et al.
Published: (2026)

Some Present-Day Problems of Romanian Library Science
by: Stoica, Ion
Published: (1973)

The Central University Library, Bucharest. Over Seventy-five Years in the History of a Collection
by: Stoica, Ion
Published: (1972)

LangProBe: a Language Programs Benchmark
by: Tan, Shangyin, et al.
Published: (2025)

Shangri-La, paraíso terrenal / Liu Huanzhi y Zhang Tao
by: Liu Huanzhi
Published: (2008)

Campesinos buscan una vida modestamente acomodada / Liu Huanzhi
by: Liu Huanzhi

vAttention: Verified Sparse Attention
by: Desai, Aditya, et al.
Published: (2025)

Benchmarking LLM-based Relevance Judgment Methods
by: Arabzadeh, Negar, et al.
Published: (2025)

Fréchet Distance for Offline Evaluation of Information Retrieval Systems with Sparse Labels
by: Arabzadeh, Negar, et al.
Published: (2024)

Offline Evaluation of Set-Based Text-to-Image Generation
by: Arabzadeh, Negar, et al.
Published: (2024)

A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment
by: Arabzadeh, Negar, et al.
Published: (2025)

A Comparison of Methods for Evaluating Generative IR
by: Arabzadeh, Negar, et al.
Published: (2024)

World Model on Million-Length Video And Language With Blockwise RingAttention
by: Liu, Hao, et al.
Published: (2024)

GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
by: Shetty, Manish, et al.
Published: (2025)

Arming shoes of the fifteenth century
by: Marquita Volken
Published: (2017)

Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs
by: Cai, Will, et al.
Published: (2025)

Can LLMs Ask Good Questions?
by: Zhang, Yueheng, et al.
Published: (2025)

The Time is Here for Just-in-Time Systems: Challenges and Opportunities
by: Liu, Shu, et al.
Published: (2026)