Recovering Mental Representations from Large Language Models with Markov Chain Monte Carlo
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929228457443328 |
|---|---|
| author | Zhu, Jian-Qiao Yan, Haijiang Griffiths, Thomas L. |
| author_facet | Zhu, Jian-Qiao Yan, Haijiang Griffiths, Thomas L. |
| contents | Simulating sampling algorithms with people has proven a useful method for efficiently probing and understanding their mental representations. We propose that the same methods can be used to study the representations of Large Language Models (LLMs). While one can always directly prompt either humans or LLMs to disclose their mental representations introspectively, we show that increased efficiency can be achieved by using LLMs as elements of a sampling algorithm. We explore the extent to which we recover human-like representations when LLMs are interrogated with Direct Sampling and Markov chain Monte Carlo (MCMC). We found a significant increase in efficiency and performance using adaptive sampling algorithms based on MCMC. We also highlight the potential of our method to yield a more general method of conducting Bayesian inference \textit{with} LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_16657 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Recovering Mental Representations from Large Language Models with Markov Chain Monte Carlo Zhu, Jian-Qiao Yan, Haijiang Griffiths, Thomas L. Artificial Intelligence Computation and Language Simulating sampling algorithms with people has proven a useful method for efficiently probing and understanding their mental representations. We propose that the same methods can be used to study the representations of Large Language Models (LLMs). While one can always directly prompt either humans or LLMs to disclose their mental representations introspectively, we show that increased efficiency can be achieved by using LLMs as elements of a sampling algorithm. We explore the extent to which we recover human-like representations when LLMs are interrogated with Direct Sampling and Markov chain Monte Carlo (MCMC). We found a significant increase in efficiency and performance using adaptive sampling algorithms based on MCMC. We also highlight the potential of our method to yield a more general method of conducting Bayesian inference \textit{with} LLMs. |
| title | Recovering Mental Representations from Large Language Models with Markov Chain Monte Carlo |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2401.16657 |