Transformer models: an introduction and catalog
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910391344300032 |
|---|---|
| author | Amatriain, Xavier Sankar, Ananth Bing, Jie Bodigutla, Praveen Kumar Hazen, Timothy J. Kazi, Michaeel |
| author_facet | Amatriain, Xavier Sankar, Ananth Bing, Jie Bodigutla, Praveen Kumar Hazen, Timothy J. Kazi, Michaeel |
| contents | In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models that are trained using self-supervised learning (e.g., BERT or GPT3) as well as those that are further trained using a human-in-the-loop (e.g. the InstructGPT model used by ChatGPT). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_07730 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Transformer models: an introduction and catalog Amatriain, Xavier Sankar, Ananth Bing, Jie Bodigutla, Praveen Kumar Hazen, Timothy J. Kazi, Michaeel Computation and Language In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models that are trained using self-supervised learning (e.g., BERT or GPT3) as well as those that are further trained using a human-in-the-loop (e.g. the InstructGPT model used by ChatGPT). |
| title | Transformer models: an introduction and catalog |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2302.07730 |