Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.19544 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908172044730368 |
|---|---|
| author | Huang, Xiang Shen, Jiayu Huang, Shanshan Cheng, Sitao Wang, Xiaxia Qu, Yuzhong |
| author_facet | Huang, Xiang Shen, Jiayu Huang, Shanshan Cheng, Sitao Wang, Xiaxia Qu, Yuzhong |
| contents | Semantic parsing, which converts natural language questions into logic forms, plays a crucial role in reasoning within structured environments. However, existing methods encounter two significant challenges: reliance on extensive manually annotated datasets and limited generalization capability to unseen examples. To tackle these issues, we propose Targeted Synthetic Data Generation (TARGA), a practical framework that dynamically generates high-relevance synthetic data without manual annotation. Starting from the pertinent entities and relations of a given question, we probe for the potential relevant queries through layer-wise expansion and cross-layer combination. Then we generate corresponding natural language questions for these constructed queries to jointly serve as the synthetic demonstrations for in-context learning. Experiments on multiple knowledge base question answering (KBQA) datasets demonstrate that TARGA, using only a 7B-parameter model, substantially outperforms existing non-fine-tuned methods that utilize close-sourced model, achieving notable improvements in F1 scores on GrailQA(+7.7) and KBQA-Agent(+12.2). Furthermore, TARGA also exhibits superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_19544 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data Huang, Xiang Shen, Jiayu Huang, Shanshan Cheng, Sitao Wang, Xiaxia Qu, Yuzhong Computation and Language Artificial Intelligence Semantic parsing, which converts natural language questions into logic forms, plays a crucial role in reasoning within structured environments. However, existing methods encounter two significant challenges: reliance on extensive manually annotated datasets and limited generalization capability to unseen examples. To tackle these issues, we propose Targeted Synthetic Data Generation (TARGA), a practical framework that dynamically generates high-relevance synthetic data without manual annotation. Starting from the pertinent entities and relations of a given question, we probe for the potential relevant queries through layer-wise expansion and cross-layer combination. Then we generate corresponding natural language questions for these constructed queries to jointly serve as the synthetic demonstrations for in-context learning. Experiments on multiple knowledge base question answering (KBQA) datasets demonstrate that TARGA, using only a 7B-parameter model, substantially outperforms existing non-fine-tuned methods that utilize close-sourced model, achieving notable improvements in F1 scores on GrailQA(+7.7) and KBQA-Agent(+12.2). Furthermore, TARGA also exhibits superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings. |
| title | TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2412.19544 |