BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911609624985600 |
|---|---|
| author | Choudhury, Deepro Williamson, Sinead Goliński, Adam Miao, Ning Smith, Freddie Bickford Kirchhof, Michael Zhang, Yizhe Rainforth, Tom |
| author_facet | Choudhury, Deepro Williamson, Sinead Goliński, Adam Miao, Ning Smith, Freddie Bickford Kirchhof, Michael Zhang, Yizhe Rainforth, Tom |
| contents | We propose a general-purpose approach for improving the ability of large language models (LLMs) to intelligently and adaptively gather information from a user or other external source using the framework of sequential Bayesian experimental design (BED). This enables LLMs to act as effective multi-turn conversational agents and interactively interface with external environments. Our approach, which we call BED-LLM (Bayesian experimental design with large language models), is based on iteratively choosing questions or queries that maximize the expected information gain (EIG) with respect to a variable of interest given the responses gathered previously. We show how this EIG can be formulated (and then estimated) in a principled way using a probabilistic model derived from the LLM's predictive distributions and provide detailed insights into key decisions in its construction and updating procedure. We find that BED-LLM achieves substantial gains in performance across a wide range of tests based on the 20 Questions game and using the LLM to actively infer user preferences, compared to purely prompting-based design generation and other adaptive design strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_21184 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design Choudhury, Deepro Williamson, Sinead Goliński, Adam Miao, Ning Smith, Freddie Bickford Kirchhof, Michael Zhang, Yizhe Rainforth, Tom Computation and Language Artificial Intelligence Machine Learning We propose a general-purpose approach for improving the ability of large language models (LLMs) to intelligently and adaptively gather information from a user or other external source using the framework of sequential Bayesian experimental design (BED). This enables LLMs to act as effective multi-turn conversational agents and interactively interface with external environments. Our approach, which we call BED-LLM (Bayesian experimental design with large language models), is based on iteratively choosing questions or queries that maximize the expected information gain (EIG) with respect to a variable of interest given the responses gathered previously. We show how this EIG can be formulated (and then estimated) in a principled way using a probabilistic model derived from the LLM's predictive distributions and provide detailed insights into key decisions in its construction and updating procedure. We find that BED-LLM achieves substantial gains in performance across a wide range of tests based on the 20 Questions game and using the LLM to actively infer user preferences, compared to purely prompting-based design generation and other adaptive design strategies. |
| title | BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Design |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2508.21184 |