A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909624633917440 |
|---|---|
| author | Sahu, Utkarsh Qi, Zhisheng Lei, Yongjia Rossi, Ryan A. Dernoncourt, Franck Ahmed, Nesreen K. Halappanavar, Mahantesh M Ma, Yao Wang, Yu |
| author_facet | Sahu, Utkarsh Qi, Zhisheng Lei, Yongjia Rossi, Ryan A. Dernoncourt, Franck Ahmed, Nesreen K. Halappanavar, Mahantesh M Ma, Yao Wang, Yu |
| contents | Large language models have been extensively studied as neural knowledge bases for their knowledge access, editability, reasoning, and explainability. However, few works focus on the structural patterns of their knowledge. Motivated by this gap, we investigate these structural patterns from a graph perspective. We quantify the knowledge of LLMs at both the triplet and entity levels, and analyze how it relates to graph structural properties such as node degree. Furthermore, we uncover the knowledge homophily, where topologically close entities exhibit similar levels of knowledgeability, which further motivates us to develop graph machine learning models to estimate entity knowledge based on its local neighbors. This model further enables valuable knowledge checking by selecting triplets less known to LLMs. Empirical results show that using selected triplets for fine-tuning leads to superior performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19286 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models Sahu, Utkarsh Qi, Zhisheng Lei, Yongjia Rossi, Ryan A. Dernoncourt, Franck Ahmed, Nesreen K. Halappanavar, Mahantesh M Ma, Yao Wang, Yu Computation and Language Machine Learning Social and Information Networks Large language models have been extensively studied as neural knowledge bases for their knowledge access, editability, reasoning, and explainability. However, few works focus on the structural patterns of their knowledge. Motivated by this gap, we investigate these structural patterns from a graph perspective. We quantify the knowledge of LLMs at both the triplet and entity levels, and analyze how it relates to graph structural properties such as node degree. Furthermore, we uncover the knowledge homophily, where topologically close entities exhibit similar levels of knowledgeability, which further motivates us to develop graph machine learning models to estimate entity knowledge based on its local neighbors. This model further enables valuable knowledge checking by selecting triplets less known to LLMs. Empirical results show that using selected triplets for fine-tuning leads to superior performance. |
| title | A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models |
| topic | Computation and Language Machine Learning Social and Information Networks |
| url | https://arxiv.org/abs/2505.19286 |