Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913888019152896 |
|---|---|
| author | Hegde, Pradyoth Kesiraju, Santosh Švec, Jan Sedláček, Šimon Yusuf, Bolaji Plchot, Oldřich T, Deepak K Černocký, Jan |
| author_facet | Hegde, Pradyoth Kesiraju, Santosh Švec, Jan Sedláček, Šimon Yusuf, Bolaji Plchot, Oldřich T, Deepak K Černocký, Jan |
| contents | This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_08753 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Factors affecting the in-context learning abilities of LLMs for dialogue state tracking Hegde, Pradyoth Kesiraju, Santosh Švec, Jan Sedláček, Šimon Yusuf, Bolaji Plchot, Oldřich T, Deepak K Černocký, Jan Computation and Language Artificial Intelligence This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking. |
| title | Factors affecting the in-context learning abilities of LLMs for dialogue state tracking |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.08753 |