I Know Therefore I Score: Label-Free Crafting of Scoring Functions using Constraints Based on Domain Expertise
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2022
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917969885396992 |
|---|---|
| author | Palakkadavath, Ragja Sivaprasad, Sarath Karande, Shirish Pedanekar, Niranjan |
| author_facet | Palakkadavath, Ragja Sivaprasad, Sarath Karande, Shirish Pedanekar, Niranjan |
| contents | Several real-life applications require crafting concise, quantitative scoring functions (also called rating systems) from measured observations. For example, an effectiveness score needs to be created for advertising campaigns using a number of engagement metrics. Experts often need to create such scoring functions in the absence of labelled data, where the scores need to reflect business insights and rules as understood by the domain experts. Without a way to capture these inputs systematically, this becomes a time-consuming process involving trial and error. In this paper, we introduce a label-free practical approach to learn a scoring function from multi-dimensional numerical data. The approach incorporates insights and business rules from domain experts in the form of easily observable and specifiable constraints, which are used as weak supervision by a machine learning model. We convert such constraints into loss functions that are optimized simultaneously while learning the scoring function. We examine the efficacy of the approach using a synthetic dataset as well as four real-life datasets, and also compare how it performs vis-a-vis supervised learning models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2203_10085 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | I Know Therefore I Score: Label-Free Crafting of Scoring Functions using Constraints Based on Domain Expertise Palakkadavath, Ragja Sivaprasad, Sarath Karande, Shirish Pedanekar, Niranjan Machine Learning Artificial Intelligence Several real-life applications require crafting concise, quantitative scoring functions (also called rating systems) from measured observations. For example, an effectiveness score needs to be created for advertising campaigns using a number of engagement metrics. Experts often need to create such scoring functions in the absence of labelled data, where the scores need to reflect business insights and rules as understood by the domain experts. Without a way to capture these inputs systematically, this becomes a time-consuming process involving trial and error. In this paper, we introduce a label-free practical approach to learn a scoring function from multi-dimensional numerical data. The approach incorporates insights and business rules from domain experts in the form of easily observable and specifiable constraints, which are used as weak supervision by a machine learning model. We convert such constraints into loss functions that are optimized simultaneously while learning the scoring function. We examine the efficacy of the approach using a synthetic dataset as well as four real-life datasets, and also compare how it performs vis-a-vis supervised learning models. |
| title | I Know Therefore I Score: Label-Free Crafting of Scoring Functions using Constraints Based on Domain Expertise |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2203.10085 |