Ensemble-Based Uncertainty Estimation for Code Correctness Estimation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915914786537472 |
|---|---|
| author | Wei, Yunxiang Li, Tianlin Zheng, Yuwei Dong, Yanni Liu, Aishan Hu, Qiang Zhang, Xiaoyu Cheng, Mingfei Yang, Jian |
| author_facet | Wei, Yunxiang Li, Tianlin Zheng, Yuwei Dong, Yanni Liu, Aishan Hu, Qiang Zhang, Xiaoyu Cheng, Mingfei Yang, Jian |
| contents | Large language models (LLMs) have demonstrated remarkable capabilities in generating programs from natural language descriptions, yet ensuring their correctness without an external oracle remains a critical challenge. To solve the challenge, existing methods often rely on uncertainty estimation, measuring the consistency of semantics or execution behaviors across multiple samples generated by a single model. However, we observe that a single model can often converge to a consistent but incorrect solution, rendering such consistency-based proxies ineffective. To address this, we propose Ensemble Semantic Entropy (ESE), which estimates uncertainty by evaluating the consistency of samples aggregated across an ensemble of models. Experiments on LiveCodeBench demonstrate that ESE correlates more strongly with program correctness than single-model semantic entropy. Notably, in selective generation tasks with strict false-positive rate constraints, ESE improves prediction accuracy by 53.4%. Furthermore, by leveraging ESE as the decision signal, we propose a cascading test-time scaling framework Cas, which maintains performance while reducing FLOPs by 64.9% compared to single-model scaling, offering a new perspective on balancing parameter and inference scaling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_27098 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Ensemble-Based Uncertainty Estimation for Code Correctness Estimation Wei, Yunxiang Li, Tianlin Zheng, Yuwei Dong, Yanni Liu, Aishan Hu, Qiang Zhang, Xiaoyu Cheng, Mingfei Yang, Jian Software Engineering Large language models (LLMs) have demonstrated remarkable capabilities in generating programs from natural language descriptions, yet ensuring their correctness without an external oracle remains a critical challenge. To solve the challenge, existing methods often rely on uncertainty estimation, measuring the consistency of semantics or execution behaviors across multiple samples generated by a single model. However, we observe that a single model can often converge to a consistent but incorrect solution, rendering such consistency-based proxies ineffective. To address this, we propose Ensemble Semantic Entropy (ESE), which estimates uncertainty by evaluating the consistency of samples aggregated across an ensemble of models. Experiments on LiveCodeBench demonstrate that ESE correlates more strongly with program correctness than single-model semantic entropy. Notably, in selective generation tasks with strict false-positive rate constraints, ESE improves prediction accuracy by 53.4%. Furthermore, by leveraging ESE as the decision signal, we propose a cascading test-time scaling framework Cas, which maintains performance while reducing FLOPs by 64.9% compared to single-model scaling, offering a new perspective on balancing parameter and inference scaling. |
| title | Ensemble-Based Uncertainty Estimation for Code Correctness Estimation |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2603.27098 |