A Collaborative Ensemble Framework for CTR Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910707103039488 |
|---|---|
| author | Liu, Xiaolong Zeng, Zhichen Liu, Xiaoyi Yuan, Siyang Song, Weinan Hang, Mengyue Liu, Yiqun Yang, Chaofei Kim, Donghyun Chen, Wen-Yen Yang, Jiyan Han, Yiping Jin, Rong Long, Bo Tong, Hanghang Yu, Philip S. |
| author_facet | Liu, Xiaolong Zeng, Zhichen Liu, Xiaoyi Yuan, Siyang Song, Weinan Hang, Mengyue Liu, Yiqun Yang, Chaofei Kim, Donghyun Chen, Wen-Yen Yang, Jiyan Han, Yiping Jin, Rong Long, Bo Tong, Hanghang Yu, Philip S. |
| contents | Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into large-scale recommendation models. However, simply increasing the model size in recommendation systems, even with large amounts of data, does not always result in the expected performance improvements. In this paper, we propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models, each with its own embedding table, to capture unique feature interaction patterns. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning, where models iteratively refine their predictions. To dynamically balance contributions from each model, we introduce a confidence-based fusion mechanism using general softmax, where model confidence is computed via negation entropy. This design ensures that more confident models have a greater influence on the final prediction while benefiting from the complementary strengths of other models. We validate our framework on three public datasets (AmazonElectronics, TaobaoAds, and KuaiVideo) as well as a large-scale industrial dataset from Meta, demonstrating its superior performance over individual models and state-of-the-art baselines. Additionally, we conduct further experiments on the Criteo and Avazu datasets to compare our method with the multi-embedding paradigm. Our results show that our framework achieves comparable or better performance with smaller embedding sizes, offering a scalable and efficient solution for CTR prediction tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_13700 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Collaborative Ensemble Framework for CTR Prediction Liu, Xiaolong Zeng, Zhichen Liu, Xiaoyi Yuan, Siyang Song, Weinan Hang, Mengyue Liu, Yiqun Yang, Chaofei Kim, Donghyun Chen, Wen-Yen Yang, Jiyan Han, Yiping Jin, Rong Long, Bo Tong, Hanghang Yu, Philip S. Information Retrieval Machine Learning Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into large-scale recommendation models. However, simply increasing the model size in recommendation systems, even with large amounts of data, does not always result in the expected performance improvements. In this paper, we propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models, each with its own embedding table, to capture unique feature interaction patterns. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning, where models iteratively refine their predictions. To dynamically balance contributions from each model, we introduce a confidence-based fusion mechanism using general softmax, where model confidence is computed via negation entropy. This design ensures that more confident models have a greater influence on the final prediction while benefiting from the complementary strengths of other models. We validate our framework on three public datasets (AmazonElectronics, TaobaoAds, and KuaiVideo) as well as a large-scale industrial dataset from Meta, demonstrating its superior performance over individual models and state-of-the-art baselines. Additionally, we conduct further experiments on the Criteo and Avazu datasets to compare our method with the multi-embedding paradigm. Our results show that our framework achieves comparable or better performance with smaller embedding sizes, offering a scalable and efficient solution for CTR prediction tasks. |
| title | A Collaborative Ensemble Framework for CTR Prediction |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2411.13700 |