Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916145268785152 |
|---|---|
| author | Rannen-Triki, Amal Bornschein, Jorg Pascanu, Razvan Hutter, Marcus György, Andras Galashov, Alexandre Teh, Yee Whye Titsias, Michalis K. |
| author_facet | Rannen-Triki, Amal Bornschein, Jorg Pascanu, Razvan Hutter, Marcus György, Andras Galashov, Alexandre Teh, Yee Whye Titsias, Michalis K. |
| contents | We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights, more in line with the concept of memory in neuroscience. We pay particular attention to the speed of adaptation (in terms of sample efficiency),sensitivity to the overall distributional drift, and the computational overhead for performing gradient computations and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and fine tuning blurs: both are methods to condition the model on previously observed tokens. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_01518 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models Rannen-Triki, Amal Bornschein, Jorg Pascanu, Razvan Hutter, Marcus György, Andras Galashov, Alexandre Teh, Yee Whye Titsias, Michalis K. Computation and Language Machine Learning We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights, more in line with the concept of memory in neuroscience. We pay particular attention to the speed of adaptation (in terms of sample efficiency),sensitivity to the overall distributional drift, and the computational overhead for performing gradient computations and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and fine tuning blurs: both are methods to condition the model on previously observed tokens. |
| title | Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2403.01518 |