Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.00353 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916510603149312 |
|---|---|
| author | Kumar, Shanu Mendke, Saish Rahman, Karody Lubna Abdul Kurasa, Santosh Agrawal, Parag Dandapat, Sandipan |
| author_facet | Kumar, Shanu Mendke, Saish Rahman, Karody Lubna Abdul Kurasa, Santosh Agrawal, Parag Dandapat, Sandipan |
| contents | Chain-of-thought (CoT) prompting has significantly enhanced the capability of large language models (LLMs) by structuring their reasoning processes. However, existing methods face critical limitations: handcrafted demonstrations require extensive human expertise, while trigger phrases are prone to inaccuracies. In this paper, we propose the Zero-shot Uncertainty-based Selection (ZEUS) method, a novel approach that improves CoT prompting by utilizing uncertainty estimates to select effective demonstrations without needing access to model parameters. Unlike traditional methods, ZEUS offers high sensitivity in distinguishing between helpful and ineffective questions, ensuring more precise and reliable selection. Our extensive evaluation shows that ZEUS consistently outperforms existing CoT strategies across four challenging reasoning benchmarks, demonstrating its robustness and scalability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00353 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection Kumar, Shanu Mendke, Saish Rahman, Karody Lubna Abdul Kurasa, Santosh Agrawal, Parag Dandapat, Sandipan Computation and Language Artificial Intelligence Chain-of-thought (CoT) prompting has significantly enhanced the capability of large language models (LLMs) by structuring their reasoning processes. However, existing methods face critical limitations: handcrafted demonstrations require extensive human expertise, while trigger phrases are prone to inaccuracies. In this paper, we propose the Zero-shot Uncertainty-based Selection (ZEUS) method, a novel approach that improves CoT prompting by utilizing uncertainty estimates to select effective demonstrations without needing access to model parameters. Unlike traditional methods, ZEUS offers high sensitivity in distinguishing between helpful and ineffective questions, ensuring more precise and reliable selection. Our extensive evaluation shows that ZEUS consistently outperforms existing CoT strategies across four challenging reasoning benchmarks, demonstrating its robustness and scalability. |
| title | Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.00353 |