Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.02603 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909887088295936 |
|---|---|
| author | Aghazadeh, Ehsan Ghasemi, Ahmad Beyhaghi, Hedyeh Pishro-Nik, Hossein |
| author_facet | Aghazadeh, Ehsan Ghasemi, Ahmad Beyhaghi, Hedyeh Pishro-Nik, Hossein |
| contents | Large language models (LLMs) are often queried multiple times at test time, with predictions aggregated by majority vote. While effective, this self-consistency strategy (arXiv:2203.11171) requires a fixed number of calls and can fail when the correct answer is rare. We introduce Confidence-Guided Early Stopping (CGES), a Bayesian framework that forms posteriors over candidate answers using scalar confidence signals derived from token probabilities or reward models. CGES adaptively halts sampling once the posterior mass of a candidate exceeds a threshold. We provide theoretical guarantees for both perfectly calibrated confidences and realistic noisy confidence signals. Across five reasoning benchmarks, CGES reduces the average number of model calls by about 69 percent (for example, from 16.0 to 4.9) while matching the accuracy of self-consistency within 0.06 percentage points. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_02603 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency Aghazadeh, Ehsan Ghasemi, Ahmad Beyhaghi, Hedyeh Pishro-Nik, Hossein Computation and Language Large language models (LLMs) are often queried multiple times at test time, with predictions aggregated by majority vote. While effective, this self-consistency strategy (arXiv:2203.11171) requires a fixed number of calls and can fail when the correct answer is rare. We introduce Confidence-Guided Early Stopping (CGES), a Bayesian framework that forms posteriors over candidate answers using scalar confidence signals derived from token probabilities or reward models. CGES adaptively halts sampling once the posterior mass of a candidate exceeds a threshold. We provide theoretical guarantees for both perfectly calibrated confidences and realistic noisy confidence signals. Across five reasoning benchmarks, CGES reduces the average number of model calls by about 69 percent (for example, from 16.0 to 4.9) while matching the accuracy of self-consistency within 0.06 percentage points. |
| title | CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.02603 |