Signature-Kernel Based Evaluation Metrics for Robust Probabilistic and Tail-Event Forecasting
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908827424653312 |
|---|---|
| author | Redhead, Benjamin R. Lee, Thomas L. Gu, Peng Elvira, Víctor Storkey, Amos |
| author_facet | Redhead, Benjamin R. Lee, Thomas L. Gu, Peng Elvira, Víctor Storkey, Amos |
| contents | Probabilistic forecasting is increasingly critical across high-stakes domains, from finance and epidemiology to climate science. However, current evaluation frameworks lack a consensus metric and suffer from two critical flaws: they often assume independence across time steps or variables, and they demonstrably lack sensitivity to tail events, the very occurrences that are most pivotal in real-world decision-making. To address these limitations, we propose two kernel-based metrics: the signature maximum mean discrepancy (Sig-MMD) and our novel censored Sig-MMD (CSig-MMD). By leveraging the signature kernel, these metrics capture complex inter-variate and inter-temporal dependencies and remain robust to missing data. Furthermore, CSig-MMD introduces a censoring scheme that prioritizes a forecaster's capability to predict tail events while strictly maintaining properness, a vital property for a good scoring rule. These metrics enable a more reliable evaluation of direct multi-step forecasting, facilitating the development of more robust probabilistic algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10182 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Signature-Kernel Based Evaluation Metrics for Robust Probabilistic and Tail-Event Forecasting Redhead, Benjamin R. Lee, Thomas L. Gu, Peng Elvira, Víctor Storkey, Amos Machine Learning 62M10 (Primary) 68T05, 62P05, 62P12 (Secondary) G.3; I.2.6; I.6.4 Probabilistic forecasting is increasingly critical across high-stakes domains, from finance and epidemiology to climate science. However, current evaluation frameworks lack a consensus metric and suffer from two critical flaws: they often assume independence across time steps or variables, and they demonstrably lack sensitivity to tail events, the very occurrences that are most pivotal in real-world decision-making. To address these limitations, we propose two kernel-based metrics: the signature maximum mean discrepancy (Sig-MMD) and our novel censored Sig-MMD (CSig-MMD). By leveraging the signature kernel, these metrics capture complex inter-variate and inter-temporal dependencies and remain robust to missing data. Furthermore, CSig-MMD introduces a censoring scheme that prioritizes a forecaster's capability to predict tail events while strictly maintaining properness, a vital property for a good scoring rule. These metrics enable a more reliable evaluation of direct multi-step forecasting, facilitating the development of more robust probabilistic algorithms. |
| title | Signature-Kernel Based Evaluation Metrics for Robust Probabilistic and Tail-Event Forecasting |
| topic | Machine Learning 62M10 (Primary) 68T05, 62P05, 62P12 (Secondary) G.3; I.2.6; I.6.4 |
| url | https://arxiv.org/abs/2602.10182 |