VeriCoT: Neuro-symbolic Chain-of-Thought Validation via Logical Consistency Checks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908634229768192 |
|---|---|
| author | Feng, Yu Weir, Nathaniel Bostrom, Kaj Bayless, Sam Cassel, Darion Chaudhary, Sapana Kiesl-Reiter, Benjamin Rangwala, Huzefa |
| author_facet | Feng, Yu Weir, Nathaniel Bostrom, Kaj Bayless, Sam Cassel, Darion Chaudhary, Sapana Kiesl-Reiter, Benjamin Rangwala, Huzefa |
| contents | LLMs can perform multi-step reasoning through Chain-of-Thought (CoT), but they cannot reliably verify their own logic. Even when they reach correct answers, the underlying reasoning may be flawed, undermining trust in high-stakes scenarios. To mitigate this issue, we introduce VeriCoT, a neuro-symbolic method that extracts and verifies formal logical arguments from CoT reasoning. VeriCoT formalizes each CoT reasoning step into first-order logic and identifies premises that ground the argument in source context, commonsense knowledge, or prior reasoning steps. The symbolic representation enables automated solvers to verify logical validity while the NL premises allow humans and systems to identify ungrounded or fallacious reasoning steps. Experiments on the ProofWriter, LegalBench, and BioASQ datasets show VeriCoT effectively identifies flawed reasoning, and serves as a strong predictor of final answer correctness. We also leverage VeriCoT's verification signal for (1) inference-time self-reflection, (2) supervised fine-tuning (SFT) on VeriCoT-distilled datasets and (3) preference fine-tuning (PFT) with direct preference optimization (DPO) using verification-based pairwise rewards, further improving reasoning validity and accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_04662 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VeriCoT: Neuro-symbolic Chain-of-Thought Validation via Logical Consistency Checks Feng, Yu Weir, Nathaniel Bostrom, Kaj Bayless, Sam Cassel, Darion Chaudhary, Sapana Kiesl-Reiter, Benjamin Rangwala, Huzefa Artificial Intelligence Computation and Language LLMs can perform multi-step reasoning through Chain-of-Thought (CoT), but they cannot reliably verify their own logic. Even when they reach correct answers, the underlying reasoning may be flawed, undermining trust in high-stakes scenarios. To mitigate this issue, we introduce VeriCoT, a neuro-symbolic method that extracts and verifies formal logical arguments from CoT reasoning. VeriCoT formalizes each CoT reasoning step into first-order logic and identifies premises that ground the argument in source context, commonsense knowledge, or prior reasoning steps. The symbolic representation enables automated solvers to verify logical validity while the NL premises allow humans and systems to identify ungrounded or fallacious reasoning steps. Experiments on the ProofWriter, LegalBench, and BioASQ datasets show VeriCoT effectively identifies flawed reasoning, and serves as a strong predictor of final answer correctness. We also leverage VeriCoT's verification signal for (1) inference-time self-reflection, (2) supervised fine-tuning (SFT) on VeriCoT-distilled datasets and (3) preference fine-tuning (PFT) with direct preference optimization (DPO) using verification-based pairwise rewards, further improving reasoning validity and accuracy. |
| title | VeriCoT: Neuro-symbolic Chain-of-Thought Validation via Logical Consistency Checks |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2511.04662 |