STEB: In Search of the Best Evaluation Approach for Synthetic Time Series
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910970673102848 |
|---|---|
| author | Stenger, Michael Leppich, Robert Bauer, André Kounev, Samuel |
| author_facet | Stenger, Michael Leppich, Robert Bauer, André Kounev, Samuel |
| contents | The growing need for synthetic time series, due to data augmentation or privacy regulations, has led to numerous generative models, frameworks, and evaluation measures alike. Objectively comparing these measures on a large scale remains an open challenge. We propose the Synthetic Time series Evaluation Benchmark (STEB) -- the first benchmark framework that enables comprehensive and interpretable automated comparisons of synthetic time series evaluation measures. Using 10 diverse datasets, randomness injection, and 13 configurable data transformations, STEB computes indicators for measure reliability and score consistency. It tracks running time, test errors, and features sequential and parallel modes of operation. In our experiments, we determine a ranking of 41 measures from literature and confirm that the choice of upstream time series embedding heavily impacts the final score. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21160 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | STEB: In Search of the Best Evaluation Approach for Synthetic Time Series Stenger, Michael Leppich, Robert Bauer, André Kounev, Samuel Machine Learning Artificial Intelligence The growing need for synthetic time series, due to data augmentation or privacy regulations, has led to numerous generative models, frameworks, and evaluation measures alike. Objectively comparing these measures on a large scale remains an open challenge. We propose the Synthetic Time series Evaluation Benchmark (STEB) -- the first benchmark framework that enables comprehensive and interpretable automated comparisons of synthetic time series evaluation measures. Using 10 diverse datasets, randomness injection, and 13 configurable data transformations, STEB computes indicators for measure reliability and score consistency. It tracks running time, test errors, and features sequential and parallel modes of operation. In our experiments, we determine a ranking of 41 measures from literature and confirm that the choice of upstream time series embedding heavily impacts the final score. |
| title | STEB: In Search of the Best Evaluation Approach for Synthetic Time Series |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2505.21160 |