Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916295979565056 |
|---|---|
| author | Choong, Han Xiang Ong, Yew-Soon Gupta, Abhishek Chen, Caishun Lim, Ray |
| author_facet | Choong, Han Xiang Ong, Yew-Soon Gupta, Abhishek Chen, Caishun Lim, Ray |
| contents | For deep learning, size is power. Massive neural nets trained on broad data for a spectrum of tasks are at the forefront of artificial intelligence. These large pre-trained models or Jacks of All Trades (JATs), when fine-tuned for downstream tasks, are gaining importance in driving deep learning advancements. However, environments with tight resource constraints, changing objectives and intentions, or varied task requirements, could limit the real-world utility of a singular JAT. Hence, in tandem with current trends towards building increasingly large JATs, this paper conducts an initial exploration into concepts underlying the creation of a diverse set of compact machine learning model sets. Composed of many smaller and specialized models, the Set of Sets is formulated to simultaneously fulfil many task settings and environmental conditions. A means to arrive at such a set tractably in one pass of a neuroevolutionary multitasking algorithm is presented for the first time, bringing us closer to models that are collectively Masters of All Trades. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2205_00671 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models Choong, Han Xiang Ong, Yew-Soon Gupta, Abhishek Chen, Caishun Lim, Ray Neural and Evolutionary Computing For deep learning, size is power. Massive neural nets trained on broad data for a spectrum of tasks are at the forefront of artificial intelligence. These large pre-trained models or Jacks of All Trades (JATs), when fine-tuned for downstream tasks, are gaining importance in driving deep learning advancements. However, environments with tight resource constraints, changing objectives and intentions, or varied task requirements, could limit the real-world utility of a singular JAT. Hence, in tandem with current trends towards building increasingly large JATs, this paper conducts an initial exploration into concepts underlying the creation of a diverse set of compact machine learning model sets. Composed of many smaller and specialized models, the Set of Sets is formulated to simultaneously fulfil many task settings and environmental conditions. A means to arrive at such a set tractably in one pass of a neuroevolutionary multitasking algorithm is presented for the first time, bringing us closer to models that are collectively Masters of All Trades. |
| title | Jack and Masters of all Trades: One-Pass Learning Sets of Model Sets From Large Pre-Trained Models |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2205.00671 |