AvalancheBench: Evaluating Enterprise Data Agents Through Latent World Recovery
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910250503766016 |
|---|---|
| author | Kleczek, Darek Zhao, Fuheng Lee, Alexander W. Tissier, Julien Liskowski, Pawel Cetintemel, Ugur Datta, Anupam |
| author_facet | Kleczek, Darek Zhao, Fuheng Lee, Alexander W. Tissier, Julien Liskowski, Pawel Cetintemel, Ugur Datta, Anupam |
| contents | We introduce AvalancheBench, a benchmark for evaluating enterprise data agents through \emph{latent world recovery}. AvalancheBench improves on existing benchmarks in three ways. First, it evaluates analytical understanding rather than pipeline completion: systems are scored on whether they recover the segments, drivers, temporal events, and relationships that explain the data, not merely on whether they execute a workflow or produce a plausible report. Second, it provides ground truth for goal-driven analytics by generating observations from a known latent world, enabling partial credit for incomplete but valid recoveries. Third, it exposes how early analytical mistakes propagate into later conclusions: missed segments, merged events, or wrong attributions can lead to systematically wrong recommendations. In this sense, AvalancheBench complements real-data benchmarks by providing a controlled setting for diagnosing whether agents recover the analytical structure behind enterprise data. On a first e-commerce use case, the strongest configuration of a leading coding agent recovers only 26\% of the rubric, with failures concentrated in generic customer segmentations and merged temporal events. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_24183 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AvalancheBench: Evaluating Enterprise Data Agents Through Latent World Recovery Kleczek, Darek Zhao, Fuheng Lee, Alexander W. Tissier, Julien Liskowski, Pawel Cetintemel, Ugur Datta, Anupam Databases Artificial Intelligence Machine Learning We introduce AvalancheBench, a benchmark for evaluating enterprise data agents through \emph{latent world recovery}. AvalancheBench improves on existing benchmarks in three ways. First, it evaluates analytical understanding rather than pipeline completion: systems are scored on whether they recover the segments, drivers, temporal events, and relationships that explain the data, not merely on whether they execute a workflow or produce a plausible report. Second, it provides ground truth for goal-driven analytics by generating observations from a known latent world, enabling partial credit for incomplete but valid recoveries. Third, it exposes how early analytical mistakes propagate into later conclusions: missed segments, merged events, or wrong attributions can lead to systematically wrong recommendations. In this sense, AvalancheBench complements real-data benchmarks by providing a controlled setting for diagnosing whether agents recover the analytical structure behind enterprise data. On a first e-commerce use case, the strongest configuration of a leading coding agent recovers only 26\% of the rubric, with failures concentrated in generic customer segmentations and merged temporal events. |
| title | AvalancheBench: Evaluating Enterprise Data Agents Through Latent World Recovery |
| topic | Databases Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2605.24183 |