Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910011068776448 |
|---|---|
| author | Lin, Xiaofeng Zhu, Sirou Chen, Yilei Chen, Mingyu Sang, Hejian Paschalidis, Ioannis Wang, Zhipeng Pacchiano, Aldo Zhang, Xuezhou |
| author_facet | Lin, Xiaofeng Zhu, Sirou Chen, Yilei Chen, Mingyu Sang, Hejian Paschalidis, Ioannis Wang, Zhipeng Pacchiano, Aldo Zhang, Xuezhou |
| contents | Large language models (LLMs) achieve strong performance when all task-relevant information is available upfront, as in static prediction and instruction-following problems. However, many real-world decision-making tasks are inherently online: crucial information must be acquired through interaction, feedback is delayed, and effective behavior requires balancing information collection and exploitation over time. While in-context learning enables adaptation without weight updates, existing LLMs often struggle to reliably leverage in-context interaction experience in such settings. In this work, we show that this limitation can be addressed through training. We introduce ORBIT, a multi-task, multi-episode meta-reinforcement learning framework that trains LLMs to learn from interaction in context. After meta-training, a relatively small open-source model (Qwen3-14B) demonstrates substantially improved in-context online learning on entirely unseen environments, matching the performance of GPT-5.2 and outperforming standard RL fine-tuning by a large margin. Scaling experiments further reveal consistent gains with model size, suggesting significant headroom for learn-at-inference-time decision-making agents. Code reproducing the results in the paper can be found at https://github.com/XiaofengLin7/ORBIT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04089 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL Lin, Xiaofeng Zhu, Sirou Chen, Yilei Chen, Mingyu Sang, Hejian Paschalidis, Ioannis Wang, Zhipeng Pacchiano, Aldo Zhang, Xuezhou Artificial Intelligence Computation and Language Machine Learning Large language models (LLMs) achieve strong performance when all task-relevant information is available upfront, as in static prediction and instruction-following problems. However, many real-world decision-making tasks are inherently online: crucial information must be acquired through interaction, feedback is delayed, and effective behavior requires balancing information collection and exploitation over time. While in-context learning enables adaptation without weight updates, existing LLMs often struggle to reliably leverage in-context interaction experience in such settings. In this work, we show that this limitation can be addressed through training. We introduce ORBIT, a multi-task, multi-episode meta-reinforcement learning framework that trains LLMs to learn from interaction in context. After meta-training, a relatively small open-source model (Qwen3-14B) demonstrates substantially improved in-context online learning on entirely unseen environments, matching the performance of GPT-5.2 and outperforming standard RL fine-tuning by a large margin. Scaling experiments further reveal consistent gains with model size, suggesting significant headroom for learn-at-inference-time decision-making agents. Code reproducing the results in the paper can be found at https://github.com/XiaofengLin7/ORBIT. |
| title | Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2602.04089 |