Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.13043 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909158957121536 |
|---|---|
| author | Lee, Seanie Cheng, Jianpeng Driesen, Joris Coca, Alexandru Johannsen, Anders |
| author_facet | Lee, Seanie Cheng, Jianpeng Driesen, Joris Coca, Alexandru Johannsen, Anders |
| contents | Few-shot dialogue state tracking (DST) with Large Language Models (LLM) relies on an effective and efficient conversation retriever to find similar in-context examples for prompt learning. Previous works use raw dialogue context as search keys and queries, and a retriever is fine-tuned with annotated dialogues to achieve superior performance. However, the approach is less suited for scaling to new domains or new annotation languages, where fine-tuning data is unavailable. To address this problem, we handle the task of conversation retrieval based on text summaries of the conversations. A LLM-based conversation summarizer is adopted for query and key generation, which enables effective maximum inner product search. To avoid the extra inference cost brought by LLM-based conversation summarization, we further distill a light-weight conversation encoder which produces query embeddings without decoding summaries for test conversations. We validate our retrieval approach on MultiWOZ datasets with GPT-Neo-2.7B and LLaMA-7B/30B. The experimental results show a significant improvement over relevant baselines in real few-shot DST settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13043 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries Lee, Seanie Cheng, Jianpeng Driesen, Joris Coca, Alexandru Johannsen, Anders Computation and Language Few-shot dialogue state tracking (DST) with Large Language Models (LLM) relies on an effective and efficient conversation retriever to find similar in-context examples for prompt learning. Previous works use raw dialogue context as search keys and queries, and a retriever is fine-tuned with annotated dialogues to achieve superior performance. However, the approach is less suited for scaling to new domains or new annotation languages, where fine-tuning data is unavailable. To address this problem, we handle the task of conversation retrieval based on text summaries of the conversations. A LLM-based conversation summarizer is adopted for query and key generation, which enables effective maximum inner product search. To avoid the extra inference cost brought by LLM-based conversation summarization, we further distill a light-weight conversation encoder which produces query embeddings without decoding summaries for test conversations. We validate our retrieval approach on MultiWOZ datasets with GPT-Neo-2.7B and LLaMA-7B/30B. The experimental results show a significant improvement over relevant baselines in real few-shot DST settings. |
| title | Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.13043 |