Evaluating Recabilities of Foundation Models: A Multi-Domain, Multi-Dataset Benchmark
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916925058056192 |
|---|---|
| author | Liu, Qijiong Zhu, Jieming Lai, Yingxin Dong, Xiaoyu Fan, Lu Bian, Zhipeng Dong, Zhenhua Wu, Xiao-Ming |
| author_facet | Liu, Qijiong Zhu, Jieming Lai, Yingxin Dong, Xiaoyu Fan, Lu Bian, Zhipeng Dong, Zhenhua Wu, Xiao-Ming |
| contents | Comprehensive evaluation of the recommendation capabilities of existing foundation models across diverse datasets and domains is essential for advancing the development of recommendation foundation models. In this study, we introduce RecBench-MD, a novel and comprehensive benchmark designed to assess the recommendation abilities of foundation models from a zero-resource, multi-dataset, and multi-domain perspective. Through extensive evaluations of 19 foundation models across 15 datasets spanning 10 diverse domains -- including e-commerce, entertainment, and social media -- we identify key characteristics of these models in recommendation tasks. Our findings suggest that in-domain fine-tuning achieves optimal performance, while cross-dataset transfer learning provides effective practical support for new recommendation scenarios. Additionally, we observe that multi-domain training significantly enhances the adaptability of foundation models. All code and data have been publicly released to facilitate future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_21354 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating Recabilities of Foundation Models: A Multi-Domain, Multi-Dataset Benchmark Liu, Qijiong Zhu, Jieming Lai, Yingxin Dong, Xiaoyu Fan, Lu Bian, Zhipeng Dong, Zhenhua Wu, Xiao-Ming Information Retrieval Comprehensive evaluation of the recommendation capabilities of existing foundation models across diverse datasets and domains is essential for advancing the development of recommendation foundation models. In this study, we introduce RecBench-MD, a novel and comprehensive benchmark designed to assess the recommendation abilities of foundation models from a zero-resource, multi-dataset, and multi-domain perspective. Through extensive evaluations of 19 foundation models across 15 datasets spanning 10 diverse domains -- including e-commerce, entertainment, and social media -- we identify key characteristics of these models in recommendation tasks. Our findings suggest that in-domain fine-tuning achieves optimal performance, while cross-dataset transfer learning provides effective practical support for new recommendation scenarios. Additionally, we observe that multi-domain training significantly enhances the adaptability of foundation models. All code and data have been publicly released to facilitate future research. |
| title | Evaluating Recabilities of Foundation Models: A Multi-Domain, Multi-Dataset Benchmark |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2508.21354 |