Crafting Interpretable Embeddings by Asking LLMs Questions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911888525230080 |
|---|---|
| author | Benara, Vinamra Singh, Chandan Morris, John X. Antonello, Richard Stoica, Ion Huth, Alexander G. Gao, Jianfeng |
| author_facet | Benara, Vinamra Singh, Chandan Morris, John X. Antonello, Richard Stoica, Ion Huth, Alexander G. Gao, Jianfeng |
| contents | Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. However, their opaqueness and proliferation into scientific domains such as neuroscience have created a growing need for interpretability. Here, we ask whether we can obtain interpretable embeddings through LLM prompting. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. Training QA-Emb reduces to selecting a set of underlying questions rather than learning model weights.
We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli. QA-Emb significantly outperforms an established interpretable baseline, and does so while requiring very few questions. This paves the way towards building flexible feature spaces that can concretize and evaluate our understanding of semantic brain representations. We additionally find that QA-Emb can be effectively approximated with an efficient model, and we explore broader applications in simple NLP tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_16714 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Crafting Interpretable Embeddings by Asking LLMs Questions Benara, Vinamra Singh, Chandan Morris, John X. Antonello, Richard Stoica, Ion Huth, Alexander G. Gao, Jianfeng Computation and Language Artificial Intelligence Machine Learning Neurons and Cognition Large language models (LLMs) have rapidly improved text embeddings for a growing array of natural-language processing tasks. However, their opaqueness and proliferation into scientific domains such as neuroscience have created a growing need for interpretability. Here, we ask whether we can obtain interpretable embeddings through LLM prompting. We introduce question-answering embeddings (QA-Emb), embeddings where each feature represents an answer to a yes/no question asked to an LLM. Training QA-Emb reduces to selecting a set of underlying questions rather than learning model weights. We use QA-Emb to flexibly generate interpretable models for predicting fMRI voxel responses to language stimuli. QA-Emb significantly outperforms an established interpretable baseline, and does so while requiring very few questions. This paves the way towards building flexible feature spaces that can concretize and evaluate our understanding of semantic brain representations. We additionally find that QA-Emb can be effectively approximated with an efficient model, and we explore broader applications in simple NLP tasks. |
| title | Crafting Interpretable Embeddings by Asking LLMs Questions |
| topic | Computation and Language Artificial Intelligence Machine Learning Neurons and Cognition |
| url | https://arxiv.org/abs/2405.16714 |