Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.04158 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912159634554880 |
|---|---|
| author | Wibiral, Tim Belaid, Mohamed Karim Rabus, Maximilian Scherp, Ansgar |
| author_facet | Wibiral, Tim Belaid, Mohamed Karim Rabus, Maximilian Scherp, Ansgar |
| contents | Assessing the importance of individual training samples is a key challenge in machine learning. Traditional approaches retrain models with and without specific samples, which is computationally expensive and ignores dependencies between data points. We introduce LossVal, an efficient data valuation method that computes importance scores during neural network training by embedding a self-weighting mechanism into loss functions like cross-entropy and mean squared error. LossVal reduces computational costs, making it suitable for large datasets and practical applications. Experiments on classification and regression tasks across multiple datasets show that LossVal effectively identifies noisy samples and is able to distinguish helpful from harmful samples. We examine the gradient calculation of LossVal to highlight its advantages. The source code is available at: https://github.com/twibiral/LossVal |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04158 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LossVal: Efficient Data Valuation for Neural Networks Wibiral, Tim Belaid, Mohamed Karim Rabus, Maximilian Scherp, Ansgar Machine Learning Assessing the importance of individual training samples is a key challenge in machine learning. Traditional approaches retrain models with and without specific samples, which is computationally expensive and ignores dependencies between data points. We introduce LossVal, an efficient data valuation method that computes importance scores during neural network training by embedding a self-weighting mechanism into loss functions like cross-entropy and mean squared error. LossVal reduces computational costs, making it suitable for large datasets and practical applications. Experiments on classification and regression tasks across multiple datasets show that LossVal effectively identifies noisy samples and is able to distinguish helpful from harmful samples. We examine the gradient calculation of LossVal to highlight its advantages. The source code is available at: https://github.com/twibiral/LossVal |
| title | LossVal: Efficient Data Valuation for Neural Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.04158 |