DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910859372003328 |
|---|---|
| author | Guo, YiQiu Yang, Yuchen Chen, Zhe Wang, Pingjie Liao, Yusheng Zhang, Ya Wang, Yanfeng Wang, Yu |
| author_facet | Guo, YiQiu Yang, Yuchen Chen, Zhe Wang, Pingjie Liao, Yusheng Zhang, Ya Wang, Yanfeng Wang, Yu |
| contents | The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models' self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD's effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_03149 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models Guo, YiQiu Yang, Yuchen Chen, Zhe Wang, Pingjie Liao, Yusheng Zhang, Ya Wang, Yanfeng Wang, Yu Computation and Language The reliability of large language models remains a critical challenge, particularly due to their susceptibility to hallucinations and factual inaccuracies during text generation. Existing solutions either underutilize models' self-correction with preemptive strategies or use costly post-hoc verification. To further explore the potential of real-time self-verification and correction, we present Dynamic Self-Verify Decoding (DSVD), a novel decoding framework that enhances generation reliability through real-time hallucination detection and efficient error correction. DSVD integrates two key components: (1) parallel self-verification architecture for continuous quality assessment, (2) dynamic rollback mechanism for targeted error recovery. Extensive experiments across five benchmarks demonstrate DSVD's effectiveness, achieving significant improvement in truthfulness (Quesetion-Answering) and factual accuracy (FActScore). Results show the DSVD can be further incorporated with existing faithful decoding methods to achieve stronger performance. Our work establishes that real-time self-verification during generation offers a viable path toward more trustworthy language models without sacrificing practical deployability. |
| title | DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.03149 |