Lost in Serialization: Invariance and Generalization of LLM Graph Reasoners
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917106335875072 |
|---|---|
| author | Herbst, Daniel Karbevska, Lea Kumar, Divyanshu Ahuja, Akanksha Nasrabadi, Fatemeh Gholamzadeh Frasca, Fabrizio |
| author_facet | Herbst, Daniel Karbevska, Lea Kumar, Divyanshu Ahuja, Akanksha Nasrabadi, Fatemeh Gholamzadeh Frasca, Fabrizio |
| contents | While promising, graph reasoners based on Large Language Models (LLMs) lack built-in invariance to symmetries in graph representations. Operating on sequential graph serializations, LLMs can produce different outputs under node reindexing, edge reordering, or formatting changes, raising robustness concerns. We systematically analyze these effects, studying how fine-tuning impacts encoding sensitivity as well generalization on unseen tasks. We propose a principled decomposition of graph serializations into node labeling, edge encoding, and syntax, and evaluate LLM robustness to variations of each of these factors on a comprehensive benchmarking suite. We also contribute a novel set of spectral tasks to further assess generalization abilities of fine-tuned reasoners. Results show that larger (non-fine-tuned) models are more robust. Fine-tuning reduces sensitivity to node relabeling but may increase it to variations in structure and format, while it does not consistently improve performance on unseen tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10234 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Lost in Serialization: Invariance and Generalization of LLM Graph Reasoners Herbst, Daniel Karbevska, Lea Kumar, Divyanshu Ahuja, Akanksha Nasrabadi, Fatemeh Gholamzadeh Frasca, Fabrizio Machine Learning Artificial Intelligence While promising, graph reasoners based on Large Language Models (LLMs) lack built-in invariance to symmetries in graph representations. Operating on sequential graph serializations, LLMs can produce different outputs under node reindexing, edge reordering, or formatting changes, raising robustness concerns. We systematically analyze these effects, studying how fine-tuning impacts encoding sensitivity as well generalization on unseen tasks. We propose a principled decomposition of graph serializations into node labeling, edge encoding, and syntax, and evaluate LLM robustness to variations of each of these factors on a comprehensive benchmarking suite. We also contribute a novel set of spectral tasks to further assess generalization abilities of fine-tuned reasoners. Results show that larger (non-fine-tuned) models are more robust. Fine-tuning reduces sensitivity to node relabeling but may increase it to variations in structure and format, while it does not consistently improve performance on unseen tasks. |
| title | Lost in Serialization: Invariance and Generalization of LLM Graph Reasoners |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.10234 |