Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909220393189376 |
|---|---|
| author | Rioux, Gabriel Nitsure, Apoorva Rigotti, Mattia Greenewald, Kristjan Mroueh, Youssef |
| author_facet | Rioux, Gabriel Nitsure, Apoorva Rigotti, Mattia Greenewald, Kristjan Mroueh, Youssef |
| contents | Stochastic dominance is an important concept in probability theory, econometrics and social choice theory for robustly modeling agents' preferences between random outcomes. While many works have been dedicated to the univariate case, little has been done in the multivariate scenario, wherein an agent has to decide between different multivariate outcomes. By exploiting a characterization of multivariate first stochastic dominance in terms of couplings, we introduce a statistic that assesses multivariate almost stochastic dominance under the framework of Optimal Transport with a smooth cost. Further, we introduce an entropic regularization of this statistic, and establish a central limit theorem (CLT) and consistency of the bootstrap procedure for the empirical statistic. Armed with this CLT, we propose a hypothesis testing framework as well as an efficient implementation using the Sinkhorn algorithm. We showcase our method in comparing and benchmarking Large Language Models that are evaluated on multiple metrics. Our multivariate stochastic dominance test allows us to capture the dependencies between the metrics in order to make an informed and statistically significant decision on the relative performance of the models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_06425 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking Rioux, Gabriel Nitsure, Apoorva Rigotti, Mattia Greenewald, Kristjan Mroueh, Youssef Machine Learning Statistics Theory Stochastic dominance is an important concept in probability theory, econometrics and social choice theory for robustly modeling agents' preferences between random outcomes. While many works have been dedicated to the univariate case, little has been done in the multivariate scenario, wherein an agent has to decide between different multivariate outcomes. By exploiting a characterization of multivariate first stochastic dominance in terms of couplings, we introduce a statistic that assesses multivariate almost stochastic dominance under the framework of Optimal Transport with a smooth cost. Further, we introduce an entropic regularization of this statistic, and establish a central limit theorem (CLT) and consistency of the bootstrap procedure for the empirical statistic. Armed with this CLT, we propose a hypothesis testing framework as well as an efficient implementation using the Sinkhorn algorithm. We showcase our method in comparing and benchmarking Large Language Models that are evaluated on multiple metrics. Our multivariate stochastic dominance test allows us to capture the dependencies between the metrics in order to make an informed and statistically significant decision on the relative performance of the models. |
| title | Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2406.06425 |