DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914832575365120 |
|---|---|
| author | Feng, Yuxi Li, Raymond Fan, Zhenan Carenini, Giuseppe Pourreza, Mohammadreza Zhang, Weiwei Zhang, Yong |
| author_facet | Feng, Yuxi Li, Raymond Fan, Zhenan Carenini, Giuseppe Pourreza, Mohammadreza Zhang, Weiwei Zhang, Yong |
| contents | While in-context Learning (ICL) has proven to be an effective technique to improve the performance of Large Language Models (LLMs) in a variety of complex tasks, notably in translating natural language questions into Structured Query Language (NL2SQL), the question of how to select the most beneficial demonstration examples remains an open research problem. While prior works often adapted off-the-shelf encoders to retrieve examples dynamically, an inherent discrepancy exists in the representational capacities between the external retrievers and the LLMs. Further, optimizing the selection of examples is a non-trivial task, since there are no straightforward methods to assess the relative benefits of examples without performing pairwise inference. To address these shortcomings, we propose DeTriever, a novel demonstration retrieval framework that learns a weighted combination of LLM hidden states, where rich semantic information is encoded. To train the model, we propose a proxy score that estimates the relative benefits of examples based on the similarities between output queries. Experiments on two popular NL2SQL benchmarks demonstrate that our method significantly outperforms the state-of-the-art baselines on one-shot NL2SQL tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07913 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning Feng, Yuxi Li, Raymond Fan, Zhenan Carenini, Giuseppe Pourreza, Mohammadreza Zhang, Weiwei Zhang, Yong Computation and Language Information Retrieval While in-context Learning (ICL) has proven to be an effective technique to improve the performance of Large Language Models (LLMs) in a variety of complex tasks, notably in translating natural language questions into Structured Query Language (NL2SQL), the question of how to select the most beneficial demonstration examples remains an open research problem. While prior works often adapted off-the-shelf encoders to retrieve examples dynamically, an inherent discrepancy exists in the representational capacities between the external retrievers and the LLMs. Further, optimizing the selection of examples is a non-trivial task, since there are no straightforward methods to assess the relative benefits of examples without performing pairwise inference. To address these shortcomings, we propose DeTriever, a novel demonstration retrieval framework that learns a weighted combination of LLM hidden states, where rich semantic information is encoded. To train the model, we propose a proxy score that estimates the relative benefits of examples based on the similarities between output queries. Experiments on two popular NL2SQL benchmarks demonstrate that our method significantly outperforms the state-of-the-art baselines on one-shot NL2SQL tasks. |
| title | DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2406.07913 |