Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.05268 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909628734898176 |
|---|---|
| author | Cazzaro, Francesco Kleindienst, Justin Gomez, Sofia Marquez Quattoni, Ariadna |
| author_facet | Cazzaro, Francesco Kleindienst, Justin Gomez, Sofia Marquez Quattoni, Ariadna |
| contents | In recent years, the need for natural language interfaces to knowledge graphs has become increasingly important since they enable easy and efficient access to the information contained in them. In particular, property graphs (PGs) have seen increased adoption as a means of representing complex structured information. Despite their growing popularity in industry, PGs remain relatively underrepresented in semantic parsing research with a lack of resources for evaluation. To address this gap, we introduce ZOGRASCOPE, a benchmark designed specifically for PGs and queries written in Cypher. Our benchmark includes a diverse set of manually annotated queries of varying complexity and is organized into three partitions: iid, compositional and length. We complement this paper with a set of experiments that test the performance of different LLMs in a variety of learning settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05268 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ZOGRASCOPE: A New Benchmark for Semantic Parsing over Property Graphs Cazzaro, Francesco Kleindienst, Justin Gomez, Sofia Marquez Quattoni, Ariadna Computation and Language In recent years, the need for natural language interfaces to knowledge graphs has become increasingly important since they enable easy and efficient access to the information contained in them. In particular, property graphs (PGs) have seen increased adoption as a means of representing complex structured information. Despite their growing popularity in industry, PGs remain relatively underrepresented in semantic parsing research with a lack of resources for evaluation. To address this gap, we introduce ZOGRASCOPE, a benchmark designed specifically for PGs and queries written in Cypher. Our benchmark includes a diverse set of manually annotated queries of varying complexity and is organized into three partitions: iid, compositional and length. We complement this paper with a set of experiments that test the performance of different LLMs in a variety of learning settings. |
| title | ZOGRASCOPE: A New Benchmark for Semantic Parsing over Property Graphs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.05268 |