Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2501.00696 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910769244798976 |
|---|---|
| author | Bhateja, Chethan O'Brien, Joseph Hashmi, Afnaan Prakash, Eva |
| author_facet | Bhateja, Chethan O'Brien, Joseph Hashmi, Afnaan Prakash, Eva |
| contents | In machine learning, metric elicitation refers to the selection of performance metrics that best reflect an individual's implicit preferences for a given application. Currently, metric elicitation methods only consider metrics that depend on the accuracy values encoded within a given model's confusion matrix. However, focusing solely on confusion matrices does not account for other model feasibility considerations such as varied monetary costs or latencies. In our work, we build upon the multiclass metric elicitation framework of Hiranandani et al., extrapolating their proposed Diagonal Linear Performance Metric Elicitation (DLPME) algorithm to account for additional bounded costs and rewards. Our experimental results with synthetic data demonstrate our approach's ability to quickly converge to the true metric. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00696 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Cost and Reward Infused Metric Elicitation Bhateja, Chethan O'Brien, Joseph Hashmi, Afnaan Prakash, Eva Machine Learning I.2.6 In machine learning, metric elicitation refers to the selection of performance metrics that best reflect an individual's implicit preferences for a given application. Currently, metric elicitation methods only consider metrics that depend on the accuracy values encoded within a given model's confusion matrix. However, focusing solely on confusion matrices does not account for other model feasibility considerations such as varied monetary costs or latencies. In our work, we build upon the multiclass metric elicitation framework of Hiranandani et al., extrapolating their proposed Diagonal Linear Performance Metric Elicitation (DLPME) algorithm to account for additional bounded costs and rewards. Our experimental results with synthetic data demonstrate our approach's ability to quickly converge to the true metric. |
| title | Cost and Reward Infused Metric Elicitation |
| topic | Machine Learning I.2.6 |
| url | https://arxiv.org/abs/2501.00696 |