Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.18771 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915756940197888 |
|---|---|
| author | Wang, Yuxiang Dai, Xinnan Fan, Wenqi Ma, Yao |
| author_facet | Wang, Yuxiang Dai, Xinnan Fan, Wenqi Ma, Yao |
| contents | In recent years, large language models (LLMs) have emerged as promising candidates for graph tasks. Many studies leverage natural language to describe graphs and apply LLMs for reasoning, yet most focus narrowly on performance benchmarks without fully comparing LLMs to graph learning models or exploring their broader potential. In this work, we present a comprehensive study of LLMs on graph learning tasks, evaluating both off-the-shelf and instruction-tuned models across a variety of scenarios. Beyond accuracy, we discuss data leakage concerns and computational overhead, and assess their performance under few-shot/zero-shot settings, domain transfer, structural understanding, and robustness. Our findings show that LLMs, particularly those with instruction tuning, greatly outperform traditional graph learning models in few-shot settings, exhibit strong domain transferability, and demonstrate excellent generalization and robustness. Our study highlights the broader capabilities of LLMs in graph learning and provides a foundation for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_18771 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation Wang, Yuxiang Dai, Xinnan Fan, Wenqi Ma, Yao Machine Learning Social and Information Networks In recent years, large language models (LLMs) have emerged as promising candidates for graph tasks. Many studies leverage natural language to describe graphs and apply LLMs for reasoning, yet most focus narrowly on performance benchmarks without fully comparing LLMs to graph learning models or exploring their broader potential. In this work, we present a comprehensive study of LLMs on graph learning tasks, evaluating both off-the-shelf and instruction-tuned models across a variety of scenarios. Beyond accuracy, we discuss data leakage concerns and computational overhead, and assess their performance under few-shot/zero-shot settings, domain transfer, structural understanding, and robustness. Our findings show that LLMs, particularly those with instruction tuning, greatly outperform traditional graph learning models in few-shot settings, exhibit strong domain transferability, and demonstrate excellent generalization and robustness. Our study highlights the broader capabilities of LLMs in graph learning and provides a foundation for future research. |
| title | Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation |
| topic | Machine Learning Social and Information Networks |
| url | https://arxiv.org/abs/2502.18771 |