VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914343996620800 |
|---|---|
| author | Zhou, Jingbo Xia, Jun Li, Siyuan Liu, Yunfan Wang, Wenjun Huang, Yufei Chi, Changxi Hong, Mutian Ouyang, Zhuoli Wang, Shu Wang, Zhongqi Wu, Xingyu Yu, Chang Li, Stan Z. |
| author_facet | Zhou, Jingbo Xia, Jun Li, Siyuan Liu, Yunfan Wang, Wenjun Huang, Yufei Chi, Changxi Hong, Mutian Ouyang, Zhuoli Wang, Shu Wang, Zhongqi Wu, Xingyu Yu, Chang Li, Stan Z. |
| contents | Graph Transformer has demonstrated impressive capabilities in the field of graph representation learning. However, existing approaches face two critical challenges: (1) most models suffer from exponentially increasing computational complexity, making it difficult to scale to large graphs; (2) attention mechanisms based on node-level operations limit the flexibility of the model and result in poor generalization performance in out-of-distribution (OOD) scenarios. To address these issues, we propose \textbf{VecFormer} (the \textbf{Vec}tor Quantized Graph Trans\textbf{former}), an efficient and highly generalizable model for node classification, particularly under OOD settings. VecFormer adopts a two-stage training paradigm. In the first stage, two codebooks are used to reconstruct the node features and the graph structure, aiming to learn the rich semantic \texttt{Graph Codes}. In the second stage, attention mechanisms are performed at the \texttt{Graph Token} level based on the transformed cross codebook, reducing computational complexity while enhancing the model's generalization capability. Extensive experiments on datasets of various sizes demonstrate that VecFormer outperforms the existing Graph Transformer in both performance and speed. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_19622 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention Zhou, Jingbo Xia, Jun Li, Siyuan Liu, Yunfan Wang, Wenjun Huang, Yufei Chi, Changxi Hong, Mutian Ouyang, Zhuoli Wang, Shu Wang, Zhongqi Wu, Xingyu Yu, Chang Li, Stan Z. Machine Learning Artificial Intelligence Graph Transformer has demonstrated impressive capabilities in the field of graph representation learning. However, existing approaches face two critical challenges: (1) most models suffer from exponentially increasing computational complexity, making it difficult to scale to large graphs; (2) attention mechanisms based on node-level operations limit the flexibility of the model and result in poor generalization performance in out-of-distribution (OOD) scenarios. To address these issues, we propose \textbf{VecFormer} (the \textbf{Vec}tor Quantized Graph Trans\textbf{former}), an efficient and highly generalizable model for node classification, particularly under OOD settings. VecFormer adopts a two-stage training paradigm. In the first stage, two codebooks are used to reconstruct the node features and the graph structure, aiming to learn the rich semantic \texttt{Graph Codes}. In the second stage, attention mechanisms are performed at the \texttt{Graph Token} level based on the transformed cross codebook, reducing computational complexity while enhancing the model's generalization capability. Extensive experiments on datasets of various sizes demonstrate that VecFormer outperforms the existing Graph Transformer in both performance and speed. |
| title | VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.19622 |