NASRec: Weight Sharing Neural Architecture Search for Recommender Systems
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917566713167872 |
|---|---|
| author | Zhang, Tunhou Cheng, Dehua He, Yuchen Chen, Zhengxing Dai, Xiaoliang Xiong, Liang Yan, Feng Li, Hai Chen, Yiran Wen, Wei |
| author_facet | Zhang, Tunhou Cheng, Dehua He, Yuchen Chen, Zhengxing Dai, Xiaoliang Xiong, Liang Yan, Feng Li, Hai Chen, Yiran Wen, Wei |
| contents | The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available at https://github.com/facebookresearch/NasRec. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2207_07187 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | NASRec: Weight Sharing Neural Architecture Search for Recommender Systems Zhang, Tunhou Cheng, Dehua He, Yuchen Chen, Zhengxing Dai, Xiaoliang Xiong, Liang Yan, Feng Li, Hai Chen, Yiran Wen, Wei Information Retrieval Machine Learning The rise of deep neural networks offers new opportunities in optimizing recommender systems. However, optimizing recommender systems using deep neural networks requires delicate architecture fabrication. We propose NASRec, a paradigm that trains a single supernet and efficiently produces abundant models/sub-architectures by weight sharing. To overcome the data multi-modality and architecture heterogeneity challenges in the recommendation domain, NASRec establishes a large supernet (i.e., search space) to search the full architectures. The supernet incorporates versatile choice of operators and dense connectivity to minimize human efforts for finding priors. The scale and heterogeneity in NASRec impose several challenges, such as training inefficiency, operator-imbalance, and degraded rank correlation. We tackle these challenges by proposing single-operator any-connection sampling, operator-balancing interaction modules, and post-training fine-tuning. Our crafted models, NASRecNet, show promising results on three Click-Through Rates (CTR) prediction benchmarks, indicating that NASRec outperforms both manually designed models and existing NAS methods with state-of-the-art performance. Our work is publicly available at https://github.com/facebookresearch/NasRec. |
| title | NASRec: Weight Sharing Neural Architecture Search for Recommender Systems |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2207.07187 |