Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909712838033408 |
|---|---|
| author | Li, Jiayu Ye, Ziyi Jian, Guohao Guo, Zhiqiang Ma, Weizhi Ai, Qingyao Zhang, Min |
| author_facet | Li, Jiayu Ye, Ziyi Jian, Guohao Guo, Zhiqiang Ma, Weizhi Ai, Qingyao Zhang, Min |
| contents | Can a recommendation model be self-aware? This paper investigates the recommender's self-awareness by quantifying its uncertainty, which provides a label-free estimation of its performance. Such self-assessment can enable more informed understanding and decision-making before the recommender engages with any users. To this end, we propose an intuitive and effective method, probability-based List Distribution uncertainty (LiDu). LiDu measures uncertainty by determining the probability that a recommender will generate a certain ranking list based on the prediction distributions of individual items. We validate LiDu's ability to represent model self-awareness in two settings: (1) with a matrix factorization model on a synthetic dataset, and (2) with popular recommendation algorithms on real-world datasets. Experimental results show that LiDu is more correlated with recommendation performance than a series of label-free performance estimators. Additionally, LiDu provides valuable insights into the dynamic inner states of models throughout training and inference. This work establishes an empirical connection between recommendation uncertainty and performance, framing it as a step towards more transparent and self-evaluating recommender systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_23208 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty Li, Jiayu Ye, Ziyi Jian, Guohao Guo, Zhiqiang Ma, Weizhi Ai, Qingyao Zhang, Min Information Retrieval Machine Learning Can a recommendation model be self-aware? This paper investigates the recommender's self-awareness by quantifying its uncertainty, which provides a label-free estimation of its performance. Such self-assessment can enable more informed understanding and decision-making before the recommender engages with any users. To this end, we propose an intuitive and effective method, probability-based List Distribution uncertainty (LiDu). LiDu measures uncertainty by determining the probability that a recommender will generate a certain ranking list based on the prediction distributions of individual items. We validate LiDu's ability to represent model self-awareness in two settings: (1) with a matrix factorization model on a synthetic dataset, and (2) with popular recommendation algorithms on real-world datasets. Experimental results show that LiDu is more correlated with recommendation performance than a series of label-free performance estimators. Additionally, LiDu provides valuable insights into the dynamic inner states of models throughout training and inference. This work establishes an empirical connection between recommendation uncertainty and performance, framing it as a step towards more transparent and self-evaluating recommender systems. |
| title | Are Recommenders Self-Aware? Label-Free Recommendation Performance Estimation via Model Uncertainty |
| topic | Information Retrieval Machine Learning |
| url | https://arxiv.org/abs/2507.23208 |