CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915987474874368 |
|---|---|
| author | Ota, Hirofumi Iwase, Naoto Ichihara, Yuki Komiyama, Junpei Imaizumi, Masaaki |
| author_facet | Ota, Hirofumi Iwase, Naoto Ichihara, Yuki Komiyama, Junpei Imaizumi, Masaaki |
| contents | Large language models often improve reasoning by sampling multiple outputs and aggregating their final answers, but precise and efficient control of error levels remains a challenging task. In particular, deciding when to stop sampling remains difficult when the stopping rule is data-dependent and the set of possible answers is not known in advance. We study anytime-valid certification of a prespecified target answer as the unique mode of the model's response distribution, a guarantee distinct from answer correctness. We propose the Certification by Intersection-union Testing with E-processes (CITE) algorithm, which provably controls false certification at any prescribed level under arbitrary data-driven stopping, without requiring prior knowledge of the answer category set. We also prove an category-set-size-free stopping-time rate, establish matching minimax lower bounds up to constants in the main regime, and extend the construction to confidence-weighted voting. Simulations and LLM self-consistency experiments show empirical error control and improved certification in diffuse-tail settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_05873 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency Ota, Hirofumi Iwase, Naoto Ichihara, Yuki Komiyama, Junpei Imaizumi, Masaaki Machine Learning Artificial Intelligence Statistics Theory Methodology Large language models often improve reasoning by sampling multiple outputs and aggregating their final answers, but precise and efficient control of error levels remains a challenging task. In particular, deciding when to stop sampling remains difficult when the stopping rule is data-dependent and the set of possible answers is not known in advance. We study anytime-valid certification of a prespecified target answer as the unique mode of the model's response distribution, a guarantee distinct from answer correctness. We propose the Certification by Intersection-union Testing with E-processes (CITE) algorithm, which provably controls false certification at any prescribed level under arbitrary data-driven stopping, without requiring prior knowledge of the answer category set. We also prove an category-set-size-free stopping-time rate, establish matching minimax lower bounds up to constants in the main regime, and extend the construction to confidence-weighted voting. Simulations and LLM self-consistency experiments show empirical error control and improved certification in diffuse-tail settings. |
| title | CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency |
| topic | Machine Learning Artificial Intelligence Statistics Theory Methodology |
| url | https://arxiv.org/abs/2605.05873 |