Handling Large-scale Cardinality in building recommendation systems
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929213212196864 |
|---|---|
| author | Kurra, Dhruva Dixith Ling, Bo Zh, Chun Ashrafzadeh, Seyedshahin |
| author_facet | Kurra, Dhruva Dixith Ling, Bo Zh, Chun Ashrafzadeh, Seyedshahin |
| contents | Effective recommendation systems rely on capturing user preferences, often requiring incorporating numerous features such as universally unique identifiers (UUIDs) of entities. However, the exceptionally high cardinality of UUIDs poses a significant challenge in terms of model degradation and increased model size due to sparsity. This paper presents two innovative techniques to address the challenge of high cardinality in recommendation systems. Specifically, we propose a bag-of-words approach, combined with layer sharing, to substantially decrease the model size while improving performance. Our techniques were evaluated through offline and online experiments on Uber use cases, resulting in promising results demonstrating our approach's effectiveness in optimizing recommendation systems and enhancing their overall performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_09572 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Handling Large-scale Cardinality in building recommendation systems Kurra, Dhruva Dixith Ling, Bo Zh, Chun Ashrafzadeh, Seyedshahin Information Retrieval Artificial Intelligence Effective recommendation systems rely on capturing user preferences, often requiring incorporating numerous features such as universally unique identifiers (UUIDs) of entities. However, the exceptionally high cardinality of UUIDs poses a significant challenge in terms of model degradation and increased model size due to sparsity. This paper presents two innovative techniques to address the challenge of high cardinality in recommendation systems. Specifically, we propose a bag-of-words approach, combined with layer sharing, to substantially decrease the model size while improving performance. Our techniques were evaluated through offline and online experiments on Uber use cases, resulting in promising results demonstrating our approach's effectiveness in optimizing recommendation systems and enhancing their overall performance. |
| title | Handling Large-scale Cardinality in building recommendation systems |
| topic | Information Retrieval Artificial Intelligence |
| url | https://arxiv.org/abs/2401.09572 |