Inverse Scaling: When Bigger Isn't Better
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916242262065152 |
|---|---|
| author | McKenzie, Ian R. Lyzhov, Alexander Pieler, Michael Parrish, Alicia Mueller, Aaron Prabhu, Ameya McLean, Euan Kirtland, Aaron Ross, Alexis Liu, Alisa Gritsevskiy, Andrew Wurgaft, Daniel Kauffman, Derik Recchia, Gabriel Liu, Jiacheng Cavanagh, Joe Weiss, Max Huang, Sicong Droid, The Floating Tseng, Tom Korbak, Tomasz Shen, Xudong Zhang, Yuhui Zhou, Zhengping Kim, Najoung Bowman, Samuel R. Perez, Ethan |
| author_facet | McKenzie, Ian R. Lyzhov, Alexander Pieler, Michael Parrish, Alicia Mueller, Aaron Prabhu, Ameya McLean, Euan Kirtland, Aaron Ross, Alexis Liu, Alisa Gritsevskiy, Andrew Wurgaft, Daniel Kauffman, Derik Recchia, Gabriel Liu, Jiacheng Cavanagh, Joe Weiss, Max Huang, Sicong Droid, The Floating Tseng, Tom Korbak, Tomasz Shen, Xudong Zhang, Yuhui Zhou, Zhengping Kim, Najoung Bowman, Samuel R. Perez, Ethan |
| contents | Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e.g., due to flaws in the training objective and data. We present empirical evidence of inverse scaling on 11 datasets collected by running a public contest, the Inverse Scaling Prize, with a substantial prize pool. Through analysis of the datasets, along with other examples found in the literature, we identify four potential causes of inverse scaling: (i) preference to repeat memorized sequences over following in-context instructions, (ii) imitation of undesirable patterns in the training data, (iii) tasks containing an easy distractor task which LMs could focus on, rather than the harder real task, and (iv) correct but misleading few-shot demonstrations of the task. We release the winning datasets at https://inversescaling.com/data to allow for further investigation of inverse scaling. Our tasks have helped drive the discovery of U-shaped and inverted-U scaling trends, where an initial trend reverses, suggesting that scaling trends are less reliable at predicting the behavior of larger-scale models than previously understood. Overall, our results suggest that there are tasks for which increased model scale alone may not lead to progress, and that more careful thought needs to go into the data and objectives for training language models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_09479 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Inverse Scaling: When Bigger Isn't Better McKenzie, Ian R. Lyzhov, Alexander Pieler, Michael Parrish, Alicia Mueller, Aaron Prabhu, Ameya McLean, Euan Kirtland, Aaron Ross, Alexis Liu, Alisa Gritsevskiy, Andrew Wurgaft, Daniel Kauffman, Derik Recchia, Gabriel Liu, Jiacheng Cavanagh, Joe Weiss, Max Huang, Sicong Droid, The Floating Tseng, Tom Korbak, Tomasz Shen, Xudong Zhang, Yuhui Zhou, Zhengping Kim, Najoung Bowman, Samuel R. Perez, Ethan Computation and Language Artificial Intelligence Computers and Society Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e.g., due to flaws in the training objective and data. We present empirical evidence of inverse scaling on 11 datasets collected by running a public contest, the Inverse Scaling Prize, with a substantial prize pool. Through analysis of the datasets, along with other examples found in the literature, we identify four potential causes of inverse scaling: (i) preference to repeat memorized sequences over following in-context instructions, (ii) imitation of undesirable patterns in the training data, (iii) tasks containing an easy distractor task which LMs could focus on, rather than the harder real task, and (iv) correct but misleading few-shot demonstrations of the task. We release the winning datasets at https://inversescaling.com/data to allow for further investigation of inverse scaling. Our tasks have helped drive the discovery of U-shaped and inverted-U scaling trends, where an initial trend reverses, suggesting that scaling trends are less reliable at predicting the behavior of larger-scale models than previously understood. Overall, our results suggest that there are tasks for which increased model scale alone may not lead to progress, and that more careful thought needs to go into the data and objectives for training language models. |
| title | Inverse Scaling: When Bigger Isn't Better |
| topic | Computation and Language Artificial Intelligence Computers and Society |
| url | https://arxiv.org/abs/2306.09479 |