<SOG_k>: One LLM Token for Explicit Graph Structural Understanding
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911416196268032 |
|---|---|
| author | Wu, Jingyao Lu, Bin Di, Zijun Gan, Xiaoying Jin, Meng Fu, Luoyi Wang, Xinbing Zhou, Chenghu |
| author_facet | Wu, Jingyao Lu, Bin Di, Zijun Gan, Xiaoying Jin, Meng Fu, Luoyi Wang, Xinbing Zhou, Chenghu |
| contents | Large language models show great potential in unstructured data understanding, but still face significant challenges with graphs due to their structural hallucination. Existing approaches mainly either verbalize graphs into natural language, which leads to excessive token consumption and scattered attention, or transform graphs into trainable continuous embeddings (i.e., soft prompt), but exhibit severe misalignment with original text tokens. To solve this problem, we propose to incorporate one special token <SOG_k> to fully represent the Structure Of Graph within a unified token space, facilitating explicit topology input and structural information sharing. Specifically, we propose a topology-aware structural tokenizer that maps each graph topology into a highly selective single token. Afterwards, we construct a set of hybrid structure Question-Answering corpora to align new structural tokens with existing text tokens. With this approach, <SOG_k> empowers LLMs to understand, generate, and reason in a concise and accurate manner. Extensive experiments on five graph-level benchmarks demonstrate the superiority of our method, achieving a performance improvement of 9.9% to 41.4% compared to the baselines while exhibiting interpretability and consistency. Furthermore, our method provides a flexible extension to node-level tasks, enabling both global and local structural understanding. The codebase is publicly available at https://github.com/Jingyao-Wu/SOG. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01771 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | <SOG_k>: One LLM Token for Explicit Graph Structural Understanding Wu, Jingyao Lu, Bin Di, Zijun Gan, Xiaoying Jin, Meng Fu, Luoyi Wang, Xinbing Zhou, Chenghu Computation and Language Artificial Intelligence Networking and Internet Architecture Large language models show great potential in unstructured data understanding, but still face significant challenges with graphs due to their structural hallucination. Existing approaches mainly either verbalize graphs into natural language, which leads to excessive token consumption and scattered attention, or transform graphs into trainable continuous embeddings (i.e., soft prompt), but exhibit severe misalignment with original text tokens. To solve this problem, we propose to incorporate one special token <SOG_k> to fully represent the Structure Of Graph within a unified token space, facilitating explicit topology input and structural information sharing. Specifically, we propose a topology-aware structural tokenizer that maps each graph topology into a highly selective single token. Afterwards, we construct a set of hybrid structure Question-Answering corpora to align new structural tokens with existing text tokens. With this approach, <SOG_k> empowers LLMs to understand, generate, and reason in a concise and accurate manner. Extensive experiments on five graph-level benchmarks demonstrate the superiority of our method, achieving a performance improvement of 9.9% to 41.4% compared to the baselines while exhibiting interpretability and consistency. Furthermore, our method provides a flexible extension to node-level tasks, enabling both global and local structural understanding. The codebase is publicly available at https://github.com/Jingyao-Wu/SOG. |
| title | <SOG_k>: One LLM Token for Explicit Graph Structural Understanding |
| topic | Computation and Language Artificial Intelligence Networking and Internet Architecture |
| url | https://arxiv.org/abs/2602.01771 |