Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913885745840128 |
|---|---|
| author | Lukasik, Michal Chen, Lin Narasimhan, Harikrishna Menon, Aditya Krishna Jitkrittum, Wittawat Yu, Felix X. Reddi, Sashank J. Fu, Gang Bateni, Mohammadhossein Kumar, Sanjiv |
| author_facet | Lukasik, Michal Chen, Lin Narasimhan, Harikrishna Menon, Aditya Krishna Jitkrittum, Wittawat Yu, Felix X. Reddi, Sashank J. Fu, Gang Bateni, Mohammadhossein Kumar, Sanjiv |
| contents | Bipartite ranking is a fundamental supervised learning problem, with the goal of learning a ranking over instances with maximal Area Under the ROC Curve (AUC) against a single binary target label. However, one may often observe multiple binary target labels, e.g., from distinct human annotators. How can one synthesize such labels into a single coherent ranking? In this work, we formally analyze two approaches to this problem -- loss aggregation and label aggregation -- by characterizing their Bayes-optimal solutions. We show that while both approaches can yield Pareto-optimal solutions, loss aggregation can exhibit label dictatorship: one can inadvertently (and undesirably) favor one label over others. This suggests that label aggregation can be preferable to loss aggregation, which we empirically verify. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_11284 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation Lukasik, Michal Chen, Lin Narasimhan, Harikrishna Menon, Aditya Krishna Jitkrittum, Wittawat Yu, Felix X. Reddi, Sashank J. Fu, Gang Bateni, Mohammadhossein Kumar, Sanjiv Machine Learning Artificial Intelligence Information Retrieval Bipartite ranking is a fundamental supervised learning problem, with the goal of learning a ranking over instances with maximal Area Under the ROC Curve (AUC) against a single binary target label. However, one may often observe multiple binary target labels, e.g., from distinct human annotators. How can one synthesize such labels into a single coherent ranking? In this work, we formally analyze two approaches to this problem -- loss aggregation and label aggregation -- by characterizing their Bayes-optimal solutions. We show that while both approaches can yield Pareto-optimal solutions, loss aggregation can exhibit label dictatorship: one can inadvertently (and undesirably) favor one label over others. This suggests that label aggregation can be preferable to loss aggregation, which we empirically verify. |
| title | Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation |
| topic | Machine Learning Artificial Intelligence Information Retrieval |
| url | https://arxiv.org/abs/2504.11284 |