Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2606.01182 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910277419663360 |
|---|---|
| author | Arnould, Daniel Aziz, Rashad Kang, Zixuan Changal, Tanav Zhu, Kevin Dev, Sunishchal Grand, Gabriel Kulkarni, Shreyas Sunil |
| author_facet | Arnould, Daniel Aziz, Rashad Kang, Zixuan Changal, Tanav Zhu, Kevin Dev, Sunishchal Grand, Gabriel Kulkarni, Shreyas Sunil |
| contents | Large Language Models (LLMs) excel at static reasoning tasks, yet their performance often degrades in interactive scenarios where information must be actively acquired through questioning. A key challenge lies in selecting questions that reduce uncertainty while incorporating responses that may be ambiguous or only partially informative. To address this, we propose Conversation-Aware Bayesian Experimental Design (CA-BED), an inference-time probabilistic dialog planning framework that integrates Bayesian Experimental Design with LLM-based likelihood estimation to optimize question selection over multiple conversational turns. CA-BED maintains a belief distribution over hypotheses, anticipates possible answers, and propagates expected information gain through a simulated conversation tree. Across two structured entity-deduction benchmarks, CA-BED yields an average 21.8% improvement in success rates over direct prompting, with comparable gains relative to alternative information-seeking methods. It achieves these gains with an average increase of only 1.8 conversational turns compared to direct prompting. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2606_01182 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CA-BED: Conversation-Aware Bayesian Experimental Design Arnould, Daniel Aziz, Rashad Kang, Zixuan Changal, Tanav Zhu, Kevin Dev, Sunishchal Grand, Gabriel Kulkarni, Shreyas Sunil Computation and Language Artificial Intelligence Large Language Models (LLMs) excel at static reasoning tasks, yet their performance often degrades in interactive scenarios where information must be actively acquired through questioning. A key challenge lies in selecting questions that reduce uncertainty while incorporating responses that may be ambiguous or only partially informative. To address this, we propose Conversation-Aware Bayesian Experimental Design (CA-BED), an inference-time probabilistic dialog planning framework that integrates Bayesian Experimental Design with LLM-based likelihood estimation to optimize question selection over multiple conversational turns. CA-BED maintains a belief distribution over hypotheses, anticipates possible answers, and propagates expected information gain through a simulated conversation tree. Across two structured entity-deduction benchmarks, CA-BED yields an average 21.8% improvement in success rates over direct prompting, with comparable gains relative to alternative information-seeking methods. It achieves these gains with an average increase of only 1.8 conversational turns compared to direct prompting. |
| title | CA-BED: Conversation-Aware Bayesian Experimental Design |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2606.01182 |