Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.18700 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915463482572800 |
|---|---|
| author | Hsu, Yi-Ping Wang, Po-Wei Eksombatchai, Chantat Xu, Jiajing |
| author_facet | Hsu, Yi-Ping Wang, Po-Wei Eksombatchai, Chantat Xu, Jiajing |
| contents | ID-based embeddings are widely used in web-scale online recommendation systems. However, their susceptibility to overfitting, particularly due to the long-tail nature of data distributions, often limits training to a single epoch, a phenomenon known as the "one-epoch problem." This challenge has driven research efforts to optimize performance within the first epoch by enhancing convergence speed or feature sparsity. In this study, we introduce a novel two-stage training strategy that incorporates a pre-training phase using a minimal model with contrastive loss, enabling broader data coverage for the embedding system. Our offline experiments demonstrate that multi-epoch training during the pre-training phase does not lead to overfitting, and the resulting embeddings improve online generalization when fine-tuned for more complex downstream recommendation tasks. We deployed the proposed system in live traffic at Pinterest, achieving significant site-wide engagement gains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_18700 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Taming the One-Epoch Phenomenon in Online Recommendation System by Two-stage Contrastive ID Pre-training Hsu, Yi-Ping Wang, Po-Wei Eksombatchai, Chantat Xu, Jiajing Information Retrieval Machine Learning ID-based embeddings are widely used in web-scale online recommendation systems. However, their susceptibility to overfitting, particularly due to the long-tail nature of data distributions, often limits training to a single epoch, a phenomenon known as the "one-epoch problem." This challenge has driven research efforts to optimize performance within the first epoch by enhancing convergence speed or feature sparsity. In this study, we introduce a novel two-stage training strategy that incorporates a pre-training phase using a minimal model with contrastive loss, enabling broader data coverage for the embedding system. Our offline experiments demonstrate that multi-epoch training during the pre-training phase does not lead to overfitting, and the resulting embeddings improve online generalization when fine-tuned for more complex downstream recommendation tasks. We deployed the proposed system in live traffic at Pinterest, achieving significant site-wide engagement gains. |
| title | Taming the One-Epoch Phenomenon in Online Recommendation System by Two-stage Contrastive ID Pre-training |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2508.18700 |