Data Efficiency for Large Recommendation Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916453628772352 |
|---|---|
| author | Jain, Kshitij Xie, Jingru Regan, Kevin Chen, Cheng Han, Jie Li, Steve Li, Zhuoshu Phillips, Todd Sussman, Myles Troup, Matt Yu, Angel Zhuo, Jia |
| author_facet | Jain, Kshitij Xie, Jingru Regan, Kevin Chen, Cheng Han, Jie Li, Steve Li, Zhuoshu Phillips, Todd Sussman, Myles Troup, Matt Yu, Angel Zhuo, Jia |
| contents | Large recommendation models (LRMs) are fundamental to the multi-billion dollar online advertising industry, processing massive datasets of hundreds of billions of examples before transitioning to continuous online training to adapt to rapidly changing user behavior. The massive scale of data directly impacts both computational costs and the speed at which new methods can be evaluated (R&D velocity). This paper presents actionable principles and high-level frameworks to guide practitioners in optimizing training data requirements. These strategies have been successfully deployed in Google's largest Ads CTR prediction models and are broadly applicable beyond LRMs. We outline the concept of data convergence, describe methods to accelerate this convergence, and finally, detail how to optimally balance training data volume with model size. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_18111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Data Efficiency for Large Recommendation Models Jain, Kshitij Xie, Jingru Regan, Kevin Chen, Cheng Han, Jie Li, Steve Li, Zhuoshu Phillips, Todd Sussman, Myles Troup, Matt Yu, Angel Zhuo, Jia Information Retrieval Machine Learning Large recommendation models (LRMs) are fundamental to the multi-billion dollar online advertising industry, processing massive datasets of hundreds of billions of examples before transitioning to continuous online training to adapt to rapidly changing user behavior. The massive scale of data directly impacts both computational costs and the speed at which new methods can be evaluated (R&D velocity). This paper presents actionable principles and high-level frameworks to guide practitioners in optimizing training data requirements. These strategies have been successfully deployed in Google's largest Ads CTR prediction models and are broadly applicable beyond LRMs. We outline the concept of data convergence, describe methods to accelerate this convergence, and finally, detail how to optimally balance training data volume with model size. |
| title | Data Efficiency for Large Recommendation Models |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2410.18111 |