Hedging and Approximate Truthfulness in Traditional Forecasting Competitions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929537359544320 |
|---|---|
| author | Monroe, Mary Thilagar, Anish Hsu, Melody Frongillo, Rafael |
| author_facet | Monroe, Mary Thilagar, Anish Hsu, Melody Frongillo, Rafael |
| contents | In forecasting competitions, the traditional mechanism scores the predictions of each contestant against the outcome of each event, and the contestant with the highest total score wins. While it is well-known that this traditional mechanism can suffer from incentive issues, it is folklore that contestants will still be roughly truthful as the number of events grows. Yet thus far the literature lacks a formal analysis of this traditional mechanism. This paper gives the first such analysis. We first demonstrate that the ''long-run truthfulness'' folklore is false: even for arbitrary numbers of events, the best forecaster can have an incentive to hedge, reporting more moderate beliefs to increase their win probability. On the positive side, however, we show that two contestants will be approximately truthful when they have sufficient uncertainty over the relative quality of their opponent and the outcomes of the events, a case which may arise in practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_19477 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Hedging and Approximate Truthfulness in Traditional Forecasting Competitions Monroe, Mary Thilagar, Anish Hsu, Melody Frongillo, Rafael Machine Learning Computer Science and Game Theory In forecasting competitions, the traditional mechanism scores the predictions of each contestant against the outcome of each event, and the contestant with the highest total score wins. While it is well-known that this traditional mechanism can suffer from incentive issues, it is folklore that contestants will still be roughly truthful as the number of events grows. Yet thus far the literature lacks a formal analysis of this traditional mechanism. This paper gives the first such analysis. We first demonstrate that the ''long-run truthfulness'' folklore is false: even for arbitrary numbers of events, the best forecaster can have an incentive to hedge, reporting more moderate beliefs to increase their win probability. On the positive side, however, we show that two contestants will be approximately truthful when they have sufficient uncertainty over the relative quality of their opponent and the outcomes of the events, a case which may arise in practice. |
| title | Hedging and Approximate Truthfulness in Traditional Forecasting Competitions |
| topic | Machine Learning Computer Science and Game Theory |
| url | https://arxiv.org/abs/2409.19477 |