Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912237197721600 |
|---|---|
| author | Shen, Tingjia Wang, Hao Wu, Chuhan Chin, Jin Yao Guo, Wei Liu, Yong Guo, Huifeng Lian, Defu Tang, Ruiming Chen, Enhong |
| author_facet | Shen, Tingjia Wang, Hao Wu, Chuhan Chin, Jin Yao Guo, Wei Liu, Yong Guo, Huifeng Lian, Defu Tang, Ruiming Chen, Enhong |
| contents | Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_00430 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy Shen, Tingjia Wang, Hao Wu, Chuhan Chin, Jin Yao Guo, Wei Liu, Yong Guo, Huifeng Lian, Defu Tang, Ruiming Chen, Enhong Artificial Intelligence Information Retrieval 68P20 H.3.4; I.2.6 Scaling Laws have emerged as a powerful framework for understanding how model performance evolves as they increase in size, providing valuable insights for optimizing computational resources. In the realm of Sequential Recommendation (SR), which is pivotal for predicting users' sequential preferences, these laws offer a lens through which to address the challenges posed by the scalability of SR models. However, the presence of structural and collaborative issues in recommender systems prevents the direct application of the Scaling Law (SL) in these systems. In response, we introduce the Performance Law for SR models, which aims to theoretically investigate and model the relationship between model performance and data quality. Specifically, we first fit the HR and NDCG metrics to transformer-based SR models. Subsequently, we propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics. Our method enables accurate predictions across various dataset scales and model sizes, demonstrating a strong correlation in large SR models and offering insights into achieving optimal performance for any given model configuration. |
| title | Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy |
| topic | Artificial Intelligence Information Retrieval 68P20 H.3.4; I.2.6 |
| url | https://arxiv.org/abs/2412.00430 |