More Permutations Do Not Always Increase Power: Non-monotonicity in Monte Carlo Permutation Tests
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915981254721536 |
|---|---|
| author | Cha, Suman Lee, Seongchan Schrab, Antonin Kim, Ilmun |
| author_facet | Cha, Suman Lee, Seongchan Schrab, Antonin Kim, Ilmun |
| contents | Monte Carlo permutation tests are a cornerstone of valid, model-free statistical inference. A widely held practical intuition is that increasing the number of sampled permutations improves test performance, in particular that statistical power tends to increase with the Monte Carlo budget. In this paper, we show that these intuitions are false in general. Leveraging the saw-toothed structure of power arising from distributional discreteness, we provide a simple structural explanation for why power can decrease as the number of sampled permutations increases, and we prove that such decreases occur infinitely often as the Monte Carlo budget grows. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_03886 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | More Permutations Do Not Always Increase Power: Non-monotonicity in Monte Carlo Permutation Tests Cha, Suman Lee, Seongchan Schrab, Antonin Kim, Ilmun Computation Statistics Theory Other Statistics Monte Carlo permutation tests are a cornerstone of valid, model-free statistical inference. A widely held practical intuition is that increasing the number of sampled permutations improves test performance, in particular that statistical power tends to increase with the Monte Carlo budget. In this paper, we show that these intuitions are false in general. Leveraging the saw-toothed structure of power arising from distributional discreteness, we provide a simple structural explanation for why power can decrease as the number of sampled permutations increases, and we prove that such decreases occur infinitely often as the Monte Carlo budget grows. |
| title | More Permutations Do Not Always Increase Power: Non-monotonicity in Monte Carlo Permutation Tests |
| topic | Computation Statistics Theory Other Statistics |
| url | https://arxiv.org/abs/2605.03886 |