Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2407.19346 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929439100633088 |
|---|---|
| author | Wilcoxson, Max Svendgård, Morten Doshi, Ria Davis, Dylan Vir, Reya Sahai, Anant |
| author_facet | Wilcoxson, Max Svendgård, Morten Doshi, Ria Davis, Dylan Vir, Reya Sahai, Anant |
| contents | Simple function classes have emerged as toy problems to better understand in-context-learning in transformer-based architectures used for large language models. But previously proposed simple function classes like linear regression or multi-layer-perceptrons lack the structure required to explore things like prompting and alignment within models capable of in-context-learning. We propose univariate polynomial regression as a function class that is just rich enough to study prompting and alignment, while allowing us to visualize and understand what is going on clearly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19346 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment Wilcoxson, Max Svendgård, Morten Doshi, Ria Davis, Dylan Vir, Reya Sahai, Anant Machine Learning Computation and Language Simple function classes have emerged as toy problems to better understand in-context-learning in transformer-based architectures used for large language models. But previously proposed simple function classes like linear regression or multi-layer-perceptrons lack the structure required to explore things like prompting and alignment within models capable of in-context-learning. We propose univariate polynomial regression as a function class that is just rich enough to study prompting and alignment, while allowing us to visualize and understand what is going on clearly. |
| title | Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2407.19346 |