Can Large Language Models Generalize Procedures Across Representations?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910010445922304 |
|---|---|
| author | Lin, Fangru Hofmann, Valentin Wan, Xingchen Wang, Weixing Ding, Zifeng Cohn, Anthony G. Pierrehumbert, Janet B. |
| author_facet | Lin, Fangru Hofmann, Valentin Wan, Xingchen Wang, Weixing Ding, Zifeng Cohn, Anthony G. Pierrehumbert, Janet B. |
| contents | Large language models (LLMs) are trained and tested extensively on symbolic representations such as code and graphs, yet real-world user tasks are often specified in natural language. To what extent can LLMs generalize across these representations? Here, we approach this question by studying isomorphic tasks involving procedures represented in code, graphs, and natural language (e.g., scheduling steps in planning). We find that training LLMs with popular post-training methods on graphs or code data alone does not reliably generalize to corresponding natural language tasks, while training solely on natural language can lead to inefficient performance gains. To address this gap, we propose a two-stage data curriculum that first trains on symbolic, then natural language data. The curriculum substantially improves model performance across model families and tasks. Remarkably, a 1.5B Qwen model trained by our method can closely match zero-shot GPT-4o in naturalistic planning. Finally, our analysis suggests that successful cross-representation generalization can be interpreted as a form of generative analogy, which our curriculum effectively encourages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_03542 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Can Large Language Models Generalize Procedures Across Representations? Lin, Fangru Hofmann, Valentin Wan, Xingchen Wang, Weixing Ding, Zifeng Cohn, Anthony G. Pierrehumbert, Janet B. Computation and Language Machine Learning Large language models (LLMs) are trained and tested extensively on symbolic representations such as code and graphs, yet real-world user tasks are often specified in natural language. To what extent can LLMs generalize across these representations? Here, we approach this question by studying isomorphic tasks involving procedures represented in code, graphs, and natural language (e.g., scheduling steps in planning). We find that training LLMs with popular post-training methods on graphs or code data alone does not reliably generalize to corresponding natural language tasks, while training solely on natural language can lead to inefficient performance gains. To address this gap, we propose a two-stage data curriculum that first trains on symbolic, then natural language data. The curriculum substantially improves model performance across model families and tasks. Remarkably, a 1.5B Qwen model trained by our method can closely match zero-shot GPT-4o in naturalistic planning. Finally, our analysis suggests that successful cross-representation generalization can be interpreted as a form of generative analogy, which our curriculum effectively encourages. |
| title | Can Large Language Models Generalize Procedures Across Representations? |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.03542 |