Scaling Laws for Pre-training Agents and World Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917871598174208 |
|---|---|
| author | Pearce, Tim Rashid, Tabish Bignell, Dave Georgescu, Raluca Devlin, Sam Hofmann, Katja |
| author_facet | Pearce, Tim Rashid, Tabish Bignell, Dave Georgescu, Raluca Devlin, Sam Hofmann, Katja |
| contents | The performance of embodied agents has been shown to improve by increasing model parameters, dataset size, and compute. This has been demonstrated in domains from robotics to video games, when generative learning objectives on offline datasets (pre-training) are used to model an agent's behavior (imitation learning) or their environment (world modeling). This paper characterizes the role of scale in these tasks more precisely. Going beyond the simple intuition that `bigger is better', we show that the same types of power laws found in language modeling also arise in world modeling and imitation learning (e.g. between loss and optimal model size). However, the coefficients of these laws are heavily influenced by the tokenizer, task \& architecture -- this has important implications on the optimal sizing of models and data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04434 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Scaling Laws for Pre-training Agents and World Models Pearce, Tim Rashid, Tabish Bignell, Dave Georgescu, Raluca Devlin, Sam Hofmann, Katja Machine Learning Artificial Intelligence The performance of embodied agents has been shown to improve by increasing model parameters, dataset size, and compute. This has been demonstrated in domains from robotics to video games, when generative learning objectives on offline datasets (pre-training) are used to model an agent's behavior (imitation learning) or their environment (world modeling). This paper characterizes the role of scale in these tasks more precisely. Going beyond the simple intuition that `bigger is better', we show that the same types of power laws found in language modeling also arise in world modeling and imitation learning (e.g. between loss and optimal model size). However, the coefficients of these laws are heavily influenced by the tokenizer, task \& architecture -- this has important implications on the optimal sizing of models and data. |
| title | Scaling Laws for Pre-training Agents and World Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2411.04434 |