Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910823992000512 |
|---|---|
| author | Lee, Nayoung Cai, Ziyang Schwarzschild, Avi Lee, Kangwook Papailiopoulos, Dimitris |
| author_facet | Lee, Nayoung Cai, Ziyang Schwarzschild, Avi Lee, Kangwook Papailiopoulos, Dimitris |
| contents | Large language models often struggle with length generalization and solving complex problem instances beyond their training distribution. We present a self-improvement approach where models iteratively generate and learn from their own solutions, progressively tackling harder problems while maintaining a standard transformer architecture. Across diverse tasks including arithmetic, string manipulation, and maze solving, self-improving enables models to solve problems far beyond their initial training distribution-for instance, generalizing from 10-digit to 100-digit addition without apparent saturation. We observe that in some cases filtering for correct self-generated examples leads to exponential improvements in out-of-distribution performance across training rounds. Additionally, starting from pretrained models significantly accelerates this self-improvement process for several tasks. Our results demonstrate how controlled weak-to-strong curricula can systematically teach a model logical extrapolation without any changes to the positional embeddings, or the model architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_01612 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges Lee, Nayoung Cai, Ziyang Schwarzschild, Avi Lee, Kangwook Papailiopoulos, Dimitris Machine Learning Artificial Intelligence Large language models often struggle with length generalization and solving complex problem instances beyond their training distribution. We present a self-improvement approach where models iteratively generate and learn from their own solutions, progressively tackling harder problems while maintaining a standard transformer architecture. Across diverse tasks including arithmetic, string manipulation, and maze solving, self-improving enables models to solve problems far beyond their initial training distribution-for instance, generalizing from 10-digit to 100-digit addition without apparent saturation. We observe that in some cases filtering for correct self-generated examples leads to exponential improvements in out-of-distribution performance across training rounds. Additionally, starting from pretrained models significantly accelerates this self-improvement process for several tasks. Our results demonstrate how controlled weak-to-strong curricula can systematically teach a model logical extrapolation without any changes to the positional embeddings, or the model architecture. |
| title | Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.01612 |