Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2509.22353 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912929006223360 |
|---|---|
| author | Wang, Fan Chen, Zhiyuan Zhong, Yuxuan Zheng, Sunjian Shao, Pengtao Yu, Bo Liu, Shaoshan Wang, Jianan Ding, Ning Cao, Yang Kang, Yu |
| author_facet | Wang, Fan Chen, Zhiyuan Zhong, Yuxuan Zheng, Sunjian Shao, Pengtao Yu, Bo Liu, Shaoshan Wang, Jianan Ding, Ning Cao, Yang Kang, Yu |
| contents | The capability of predicting environmental dynamics underpins both biological neural systems and general embodied AI in adapting to their surroundings. Yet prevailing approaches rest on static world models that falter when confronted with novel or rare configurations. We investigate in-context learning (ICL) of world models, shifting attention from zero-shot performance to the growth and asymptotic limits of the world model. Our contributions are three-fold: (1) we formalize ICL of a world model and identify two core mechanisms: environment recognition (ER) and environment learning (EL); (2) we derive error upper-bounds for both mechanisms that expose how the mechanisms emerge; and (3) we empirically confirm that distinct ICL mechanisms exist in the world model, and we further investigate how data distribution and model architecture affect ICL in a manner consistent with theory. These findings demonstrate the potential of self-adapting world models and highlight the key factors behind the emergence of EL/ER, most notably the necessity of long context and diverse environments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_22353 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Context and Diversity Matter: The Emergence of In-Context Learning in World Models Wang, Fan Chen, Zhiyuan Zhong, Yuxuan Zheng, Sunjian Shao, Pengtao Yu, Bo Liu, Shaoshan Wang, Jianan Ding, Ning Cao, Yang Kang, Yu Machine Learning Artificial Intelligence The capability of predicting environmental dynamics underpins both biological neural systems and general embodied AI in adapting to their surroundings. Yet prevailing approaches rest on static world models that falter when confronted with novel or rare configurations. We investigate in-context learning (ICL) of world models, shifting attention from zero-shot performance to the growth and asymptotic limits of the world model. Our contributions are three-fold: (1) we formalize ICL of a world model and identify two core mechanisms: environment recognition (ER) and environment learning (EL); (2) we derive error upper-bounds for both mechanisms that expose how the mechanisms emerge; and (3) we empirically confirm that distinct ICL mechanisms exist in the world model, and we further investigate how data distribution and model architecture affect ICL in a manner consistent with theory. These findings demonstrate the potential of self-adapting world models and highlight the key factors behind the emergence of EL/ER, most notably the necessity of long context and diverse environments. |
| title | Context and Diversity Matter: The Emergence of In-Context Learning in World Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.22353 |