Let Your Graph Do the Talking: Encoding Structured Data for LLMs
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866929237964881920 |
|---|---|
| author | Perozzi, Bryan Fatemi, Bahare Zelle, Dustin Tsitsulin, Anton Kazemi, Mehran Al-Rfou, Rami Halcrow, Jonathan |
| author_facet | Perozzi, Bryan Fatemi, Bahare Zelle, Dustin Tsitsulin, Anton Kazemi, Mehran Al-Rfou, Rami Halcrow, Jonathan |
| contents | How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent structured data for LLMs. Our method, GraphToken, learns an encoding function to extend prompts with explicit structured information. Unlike other work which focuses on limited domains (e.g. knowledge graph representation), our work is the first effort focused on the general encoding of structured data to be used for various reasoning tasks. We show that explicitly representing the graph structure allows significant improvements to graph reasoning tasks. Specifically, we see across the board improvements - up to 73% points - on node, edge and, graph-level tasks from the GraphQA benchmark. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_05862 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Let Your Graph Do the Talking: Encoding Structured Data for LLMs Perozzi, Bryan Fatemi, Bahare Zelle, Dustin Tsitsulin, Anton Kazemi, Mehran Al-Rfou, Rami Halcrow, Jonathan Machine Learning Artificial Intelligence Social and Information Networks I.5.1; I.2.6; I.2.7 How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent structured data for LLMs. Our method, GraphToken, learns an encoding function to extend prompts with explicit structured information. Unlike other work which focuses on limited domains (e.g. knowledge graph representation), our work is the first effort focused on the general encoding of structured data to be used for various reasoning tasks. We show that explicitly representing the graph structure allows significant improvements to graph reasoning tasks. Specifically, we see across the board improvements - up to 73% points - on node, edge and, graph-level tasks from the GraphQA benchmark. |
| title | Let Your Graph Do the Talking: Encoding Structured Data for LLMs |
| topic | Machine Learning Artificial Intelligence Social and Information Networks I.5.1; I.2.6; I.2.7 |
| url | https://arxiv.org/abs/2402.05862 |