STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914875262894080 |
|---|---|
| author | Zolnai-Lucas, Aaron Boylan, Jack Hokamp, Chris Ghaffari, Parsa |
| author_facet | Zolnai-Lucas, Aaron Boylan, Jack Hokamp, Chris Ghaffari, Parsa |
| contents | We present Simplified Text-Attributed Graph Embeddings (STAGE), a straightforward yet effective method for enhancing node features in Graph Neural Network (GNN) models that encode Text-Attributed Graphs (TAGs). Our approach leverages Large-Language Models (LLMs) to generate embeddings for textual attributes. STAGE achieves competitive results on various node classification benchmarks while also maintaining a simplicity in implementation relative to current state-of-the-art (SoTA) techniques. We show that utilizing pre-trained LLMs as embedding generators provides robust features for ensemble GNN training, enabling pipelines that are simpler than current SoTA approaches which require multiple expensive training and prompting stages. We also implement diffusion-pattern GNNs in an effort to make this pipeline scalable to graphs beyond academic benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12860 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs Zolnai-Lucas, Aaron Boylan, Jack Hokamp, Chris Ghaffari, Parsa Computation and Language Artificial Intelligence We present Simplified Text-Attributed Graph Embeddings (STAGE), a straightforward yet effective method for enhancing node features in Graph Neural Network (GNN) models that encode Text-Attributed Graphs (TAGs). Our approach leverages Large-Language Models (LLMs) to generate embeddings for textual attributes. STAGE achieves competitive results on various node classification benchmarks while also maintaining a simplicity in implementation relative to current state-of-the-art (SoTA) techniques. We show that utilizing pre-trained LLMs as embedding generators provides robust features for ensemble GNN training, enabling pipelines that are simpler than current SoTA approaches which require multiple expensive training and prompting stages. We also implement diffusion-pattern GNNs in an effort to make this pipeline scalable to graphs beyond academic benchmarks. |
| title | STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2407.12860 |