DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914241585348608 |
|---|---|
| author | Xiong, Jing Li, Zixuan Zheng, Chuanyang Guo, Zhijiang Yin, Yichun Xie, Enze Yang, Zhicheng Cao, Qingxing Wang, Haiming Han, Xiongwei Tang, Jing Li, Chengming Liang, Xiaodan |
| author_facet | Xiong, Jing Li, Zixuan Zheng, Chuanyang Guo, Zhijiang Yin, Yichun Xie, Enze Yang, Zhicheng Cao, Qingxing Wang, Haiming Han, Xiongwei Tang, Jing Li, Chengming Liang, Xiaodan |
| contents | Recent advances in natural language processing, primarily propelled by Large Language Models (LLMs), have showcased their remarkable capabilities grounded in in-context learning. A promising avenue for guiding LLMs in intricate reasoning tasks involves the utilization of intermediate reasoning steps within the Chain-of-Thought (CoT) paradigm. Nevertheless, the central challenge lies in the effective selection of exemplars for facilitating in-context learning. In this study, we introduce a framework that leverages Dual Queries and Low-rank approximation Re-ranking (DQ-LoRe) to automatically select exemplars for in-context learning. Dual Queries first query LLM to obtain LLM-generated knowledge such as CoT, then query the retriever to obtain the final exemplars via both question and the knowledge. Moreover, for the second query, LoRe employs dimensionality reduction techniques to refine exemplar selection, ensuring close alignment with the input question's knowledge. Through extensive experiments, we demonstrate that DQ-LoRe significantly outperforms prior state-of-the-art methods in the automatic selection of exemplars for GPT-4, enhancing performance from 92.5% to 94.2%. Our comprehensive analysis further reveals that DQ-LoRe consistently outperforms retrieval-based approaches in terms of both performance and adaptability, especially in scenarios characterized by distribution shifts. DQ-LoRe pushes the boundary of in-context learning and opens up new avenues for addressing complex reasoning challenges. Our code is released at https://github.com/menik1126/DQ-LoRe |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_02954 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning Xiong, Jing Li, Zixuan Zheng, Chuanyang Guo, Zhijiang Yin, Yichun Xie, Enze Yang, Zhicheng Cao, Qingxing Wang, Haiming Han, Xiongwei Tang, Jing Li, Chengming Liang, Xiaodan Computation and Language Recent advances in natural language processing, primarily propelled by Large Language Models (LLMs), have showcased their remarkable capabilities grounded in in-context learning. A promising avenue for guiding LLMs in intricate reasoning tasks involves the utilization of intermediate reasoning steps within the Chain-of-Thought (CoT) paradigm. Nevertheless, the central challenge lies in the effective selection of exemplars for facilitating in-context learning. In this study, we introduce a framework that leverages Dual Queries and Low-rank approximation Re-ranking (DQ-LoRe) to automatically select exemplars for in-context learning. Dual Queries first query LLM to obtain LLM-generated knowledge such as CoT, then query the retriever to obtain the final exemplars via both question and the knowledge. Moreover, for the second query, LoRe employs dimensionality reduction techniques to refine exemplar selection, ensuring close alignment with the input question's knowledge. Through extensive experiments, we demonstrate that DQ-LoRe significantly outperforms prior state-of-the-art methods in the automatic selection of exemplars for GPT-4, enhancing performance from 92.5% to 94.2%. Our comprehensive analysis further reveals that DQ-LoRe consistently outperforms retrieval-based approaches in terms of both performance and adaptability, especially in scenarios characterized by distribution shifts. DQ-LoRe pushes the boundary of in-context learning and opens up new avenues for addressing complex reasoning challenges. Our code is released at https://github.com/menik1126/DQ-LoRe |
| title | DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2310.02954 |