LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917398345416704 |
|---|---|
| author | Gajo, Paolo Rosati, Domenic Sajjad, Hassan Barrón-Cedeño, Alberto |
| author_facet | Gajo, Paolo Rosati, Domenic Sajjad, Hassan Barrón-Cedeño, Alberto |
| contents | Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_08752 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs Gajo, Paolo Rosati, Domenic Sajjad, Hassan Barrón-Cedeño, Alberto Computation and Language Artificial Intelligence Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs. |
| title | LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2604.08752 |