On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913086844174336 |
|---|---|
| author | Kim, Sunghwan Cho, Junhee Kwak, Beong-woo Kwon, Taeyoon Wang, Liang Yang, Nan Zhang, Xingxing Wei, Furu Yeo, Jinyoung |
| author_facet | Kim, Sunghwan Cho, Junhee Kwak, Beong-woo Kwon, Taeyoon Wang, Liang Yang, Nan Zhang, Xingxing Wei, Furu Yeo, Jinyoung |
| contents | Large language models (LLMs) have shown promise as interactive agents that solve tasks through extended sequences of environment interactions. While prior work has primarily focused on system-level optimizations or algorithmic improvements, the role of task horizon length in shaping training dynamics remains poorly understood. In this work, we present a systematic empirical study that examines horizon length through controlled task constructions. Specifically, we construct controlled tasks in which agents face identical decision rules and reasoning structures, but differ only in the length of action sequences required for successful completion. Our results reveal that increasing horizon length alone constitutes a training bottleneck, inducing severe training instability driven by exploration difficulties and credit assignment challenges. We demonstrate that horizon reduction is a key principle to address this limitation, stabilizing training and achieving better performance in long-horizon tasks. Moreover, we find that horizon reduction is related to stronger generalization across horizon lengths: models trained under reduced horizons generalize more effectively to longer-horizon variants at inference time, a phenomenon we refer to as horizon generalization. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02572 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length Kim, Sunghwan Cho, Junhee Kwak, Beong-woo Kwon, Taeyoon Wang, Liang Yang, Nan Zhang, Xingxing Wei, Furu Yeo, Jinyoung Artificial Intelligence Machine Learning Large language models (LLMs) have shown promise as interactive agents that solve tasks through extended sequences of environment interactions. While prior work has primarily focused on system-level optimizations or algorithmic improvements, the role of task horizon length in shaping training dynamics remains poorly understood. In this work, we present a systematic empirical study that examines horizon length through controlled task constructions. Specifically, we construct controlled tasks in which agents face identical decision rules and reasoning structures, but differ only in the length of action sequences required for successful completion. Our results reveal that increasing horizon length alone constitutes a training bottleneck, inducing severe training instability driven by exploration difficulties and credit assignment challenges. We demonstrate that horizon reduction is a key principle to address this limitation, stabilizing training and achieving better performance in long-horizon tasks. Moreover, we find that horizon reduction is related to stronger generalization across horizon lengths: models trained under reduced horizons generalize more effectively to longer-horizon variants at inference time, a phenomenon we refer to as horizon generalization. |
| title | On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2605.02572 |