Guardado en:
| Autores principales: | , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2605.24703 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917528380375040 |
|---|---|
| author | Han, Liying Yang, Kang Wang, Oliver Wu, Jason Quan, Pengrui Dong, Gaofeng Mulayim, Ozan Baris Ma, Sizhe Yuan, Yuyang Hong, Dezhi Berges, Mario Srivastava, Mani |
| author_facet | Han, Liying Yang, Kang Wang, Oliver Wu, Jason Quan, Pengrui Dong, Gaofeng Mulayim, Ozan Baris Ma, Sizhe Yuan, Yuyang Hong, Dezhi Berges, Mario Srivastava, Mani |
| contents | Large language models (LLMs) and time-series language models (TSLMs) are increasingly applied to time-series question answering (TSQA). Unlike text-only QA, TSQA requires models to ground answers in temporal signals whose patterns may occur at different scales, specific time locations, or across separated intervals. However, existing benchmarks are typically organized by task types or high-level reasoning categories, making it difficult to diagnose the underlying signal-level capabilities driving model performance. We introduce TS-Skill, a controlled benchmark for evaluating three composable analytical skills in TSQA: temporal scale selection (SK1), temporal localization (SK2), and cross-interval integration (SK3). TS-Skill provides timestamp-aware questions, broad domain coverage, and human-validated QA quality. To construct the benchmark at scale, we develop SKEvol, a skill-guided agentic framework that combines domain-aware time-series seed generation, skill-controlled question generation, metadata- and code-assisted answer construction, multi-phase signal-grounded verification, and human-in-the-loop curation. Experiments on ten state-of-the-art LLMs and TSLMs reveal substantial and uneven capability gaps across SK1-SK3. In particular, SK3 remains consistently challenging for non-agent models, whereas tool-augmented agents show a selective advantage on standalone SK3. These findings demonstrate that skill-level evaluation can uncover temporal reasoning failures that are obscured by aggregate TSQA scores. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_24703 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TS-Skill: A Benchmark for Evaluating Analytical Skills in Time-Series Question Answering Han, Liying Yang, Kang Wang, Oliver Wu, Jason Quan, Pengrui Dong, Gaofeng Mulayim, Ozan Baris Ma, Sizhe Yuan, Yuyang Hong, Dezhi Berges, Mario Srivastava, Mani Computation and Language Artificial Intelligence Large language models (LLMs) and time-series language models (TSLMs) are increasingly applied to time-series question answering (TSQA). Unlike text-only QA, TSQA requires models to ground answers in temporal signals whose patterns may occur at different scales, specific time locations, or across separated intervals. However, existing benchmarks are typically organized by task types or high-level reasoning categories, making it difficult to diagnose the underlying signal-level capabilities driving model performance. We introduce TS-Skill, a controlled benchmark for evaluating three composable analytical skills in TSQA: temporal scale selection (SK1), temporal localization (SK2), and cross-interval integration (SK3). TS-Skill provides timestamp-aware questions, broad domain coverage, and human-validated QA quality. To construct the benchmark at scale, we develop SKEvol, a skill-guided agentic framework that combines domain-aware time-series seed generation, skill-controlled question generation, metadata- and code-assisted answer construction, multi-phase signal-grounded verification, and human-in-the-loop curation. Experiments on ten state-of-the-art LLMs and TSLMs reveal substantial and uneven capability gaps across SK1-SK3. In particular, SK3 remains consistently challenging for non-agent models, whereas tool-augmented agents show a selective advantage on standalone SK3. These findings demonstrate that skill-level evaluation can uncover temporal reasoning failures that are obscured by aggregate TSQA scores. |
| title | TS-Skill: A Benchmark for Evaluating Analytical Skills in Time-Series Question Answering |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2605.24703 |