Rethinking Semantic Parsing for Large Language Models: Enhancing LLM Performance with Semantic Hints
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915306601971712 |
|---|---|
| author | An, Kaikai Si, Shuzheng Hu, Helan Zhao, Haozhe Wang, Yuchi Guo, Qingyan Chang, Baobao |
| author_facet | An, Kaikai Si, Shuzheng Hu, Helan Zhao, Haozhe Wang, Yuchi Guo, Qingyan Chang, Baobao |
| contents | Semantic Parsing aims to capture the meaning of a sentence and convert it into a logical, structured form. Previous studies show that semantic parsing enhances the performance of smaller models (e.g., BERT) on downstream tasks. However, it remains unclear whether the improvements extend similarly to LLMs. In this paper, our empirical findings reveal that, unlike smaller models, directly adding semantic parsing results into LLMs reduces their performance. To overcome this, we propose SENSE, a novel prompting approach that embeds semantic hints within the prompt. Experiments show that SENSE consistently improves LLMs' performance across various tasks, highlighting the potential of integrating semantic information to improve LLM capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_14469 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Rethinking Semantic Parsing for Large Language Models: Enhancing LLM Performance with Semantic Hints An, Kaikai Si, Shuzheng Hu, Helan Zhao, Haozhe Wang, Yuchi Guo, Qingyan Chang, Baobao Computation and Language Semantic Parsing aims to capture the meaning of a sentence and convert it into a logical, structured form. Previous studies show that semantic parsing enhances the performance of smaller models (e.g., BERT) on downstream tasks. However, it remains unclear whether the improvements extend similarly to LLMs. In this paper, our empirical findings reveal that, unlike smaller models, directly adding semantic parsing results into LLMs reduces their performance. To overcome this, we propose SENSE, a novel prompting approach that embeds semantic hints within the prompt. Experiments show that SENSE consistently improves LLMs' performance across various tasks, highlighting the potential of integrating semantic information to improve LLM capabilities. |
| title | Rethinking Semantic Parsing for Large Language Models: Enhancing LLM Performance with Semantic Hints |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.14469 |