Cover Image

Toward a Graph Foundation Model: Pre-Training Transformers With Random Walks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tang, Ziyuan, Chen, Jie
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Revisiting Random Walks for Learning on Graphs
by: Kim, Jinwoo, et al.
Published: (2024)

Towards Graph Foundation Models: Training on Knowledge Graphs Enables Transferability to General Graphs
by: Wang, Kai, et al.
Published: (2024)

Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoders
by: Lei, Zhuxin, et al.
Published: (2026)

Topology Only Pre-Training: Towards Generalised Multi-Domain Graph Models
by: Davies, Alex O., et al.
Published: (2023)

Toward Understanding BERT-Like Pre-Training for DNA Foundation Models
by: Liang, Chaoqi, et al.
Published: (2023)

Graph Neural Networks with Feature and Structure Aware Random Walk
by: Zhuo, Wei, et al.
Published: (2021)

Distance-Misaligned Training in Graph Transformers and Adaptive Graph-Aware Control
by: Hou, Qinhan, et al.
Published: (2026)

GraphProp: Training the Graph Foundation Models using Graph Properties
by: Sun, Ziheng, et al.
Published: (2025)

Graph Generative Pre-trained Transformer
by: Chen, Xiaohui, et al.
Published: (2025)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
by: Tang, Anke, et al.
Published: (2024)

TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification
by: Yeh, Chin-Chia Michael, et al.
Published: (2025)

GraphGPT: Generative Pre-trained Graph Eulerian Transformer
by: Zhao, Qifang, et al.
Published: (2023)

Mochi: Aligning Pre-training and Inference for Efficient Graph Foundation Models via Meta-Learning
by: Mattos, João, et al.
Published: (2026)

GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs
by: Zhu, Yun, et al.
Published: (2024)

Free Lunch in Medical Image Foundation Model Pre-training via Randomized Synthesis and Disentanglement
by: Wei, Yuhan, et al.
Published: (2026)

Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
by: Hübotter, Jonas, et al.
Published: (2025)

MSAGPT: Neural Prompting Protein Structure Prediction via MSA Generative Pre-Training
by: Chen, Bo, et al.
Published: (2024)

Griffin: Towards a Graph-Centric Relational Database Foundation Model
by: Wang, Yanbo, et al.
Published: (2025)

Towards Principled Graph Transformers
by: Müller, Luis, et al.
Published: (2024)

On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models
by: Farhat, Sean, et al.
Published: (2024)

Single Parent Family: A Spectrum of Family Members from a Single Pre-Trained Foundation Model
by: Hajimolahoseini, Habib, et al.
Published: (2024)

Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
by: Zhao, Huanjing, et al.
Published: (2024)

Towards a Physics Foundation Model
by: Wiesner, Florian, et al.
Published: (2025)

OpenGraph: Towards Open Graph Foundation Models
by: Xia, Lianghao, et al.
Published: (2024)

OSF: On Pre-training and Scaling of Sleep Foundation Models
by: Shuai, Zitao, et al.
Published: (2026)

Foundations and Frontiers of Graph Learning Theory
by: Huang, Yu, et al.
Published: (2024)

AnyGraph: Graph Foundation Model in the Wild
by: Xia, Lianghao, et al.
Published: (2024)

Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data
by: Ranjan, Rishabh, et al.
Published: (2025)

Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG
by: Wei, Xinxu, et al.
Published: (2024)

The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training
by: Wang, Jinbo, et al.
Published: (2025)

Towards Mechanistic Interpretability of Graph Transformers via Attention Graphs
by: El, Batu, et al.
Published: (2025)

Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning
by: Li, Yanjie, et al.
Published: (2024)

TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism
by: Chen, Hongjiang, et al.
Published: (2026)

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training
by: Zheng, Ziyu, et al.
Published: (2026)

A Time Series Multitask Framework Integrating a Large Language Model, Pre-Trained Time Series Model, and Knowledge Graph
by: Hao, Shule, et al.
Published: (2025)

Foundations of Schrödinger Bridges for Generative Modeling
by: Tang, Sophia
Published: (2026)

Billion-Scale Graph Foundation Models
by: Bechler-Speicher, Maya, et al.
Published: (2026)

PTMs-TSCIL Pre-Trained Models Based Class-Incremental Learning
by: Wu, Yuanlong, et al.
Published: (2025)

Towards Physics-Guided Foundation Models
by: Farhadloo, Majid, et al.
Published: (2025)

Graph Foundation Models: Bridging Language Model Paradigms and Graph Optimization
by: Liang, Yunhao, et al.
Published: (2025)