RankDCG: Rank-Ordering Evaluation Measure
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2018
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911655419445248 |
|---|---|
| author | Katerenchuk, Denys Rosenberg, Andrew |
| author_facet | Katerenchuk, Denys Rosenberg, Andrew |
| contents | Ranking is used for a wide array of problems, most notably information retrieval (search). There are a number of popular approaches to the evaluation of ranking such as Kendall's $τ$, Average Precision, and nDCG. When dealing with problems such as user ranking or recommendation systems, all these measures suffer from various problems, including an inability to deal with elements of the same rank, inconsistent and ambiguous lower bound scores, and an inappropriate cost function. We propose a new measure, rankDCG, that addresses these problems. This is a modification of the popular nDCG algorithm. We provide a number of criteria for any effective ranking algorithm and show that only rankDCG satisfies all of them. Results are presented on constructed and real data sets. We release a publicly available rankDCG evaluation package. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1803_00719 |
| institution | arXiv |
| publishDate | 2018 |
| record_format | arxiv |
| spellingShingle | RankDCG: Rank-Ordering Evaluation Measure Katerenchuk, Denys Rosenberg, Andrew Information Retrieval Social and Information Networks Ranking is used for a wide array of problems, most notably information retrieval (search). There are a number of popular approaches to the evaluation of ranking such as Kendall's $τ$, Average Precision, and nDCG. When dealing with problems such as user ranking or recommendation systems, all these measures suffer from various problems, including an inability to deal with elements of the same rank, inconsistent and ambiguous lower bound scores, and an inappropriate cost function. We propose a new measure, rankDCG, that addresses these problems. This is a modification of the popular nDCG algorithm. We provide a number of criteria for any effective ranking algorithm and show that only rankDCG satisfies all of them. Results are presented on constructed and real data sets. We release a publicly available rankDCG evaluation package. |
| title | RankDCG: Rank-Ordering Evaluation Measure |
| topic | Information Retrieval Social and Information Networks |
| url | https://arxiv.org/abs/1803.00719 |