Reducing Instability in Synthetic Data Evaluation with a Super-Metric in MalDataGen
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917095992721408 |
|---|---|
| author | da Silva, Anna Luiza Gomes Kreutz, Diego Diniz, Angelo Mansilha, Rodrigo da Fonseca, Celso Nobre |
| author_facet | da Silva, Anna Luiza Gomes Kreutz, Diego Diniz, Angelo Mansilha, Rodrigo da Fonseca, Celso Nobre |
| contents | Evaluating the quality of synthetic data remains a persistent challenge in the Android malware domain due to instability and the lack of standardization among existing metrics. This work integrates into MalDataGen a Super-Metric that aggregates eight metrics across four fidelity dimensions, producing a single weighted score. Experiments involving ten generative models and five balanced datasets demonstrate that the Super-Metric is more stable and consistent than traditional metrics, exhibiting stronger correlations with the actual performance of classifiers. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_16373 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reducing Instability in Synthetic Data Evaluation with a Super-Metric in MalDataGen da Silva, Anna Luiza Gomes Kreutz, Diego Diniz, Angelo Mansilha, Rodrigo da Fonseca, Celso Nobre Artificial Intelligence Machine Learning 68T01 I.2 Evaluating the quality of synthetic data remains a persistent challenge in the Android malware domain due to instability and the lack of standardization among existing metrics. This work integrates into MalDataGen a Super-Metric that aggregates eight metrics across four fidelity dimensions, producing a single weighted score. Experiments involving ten generative models and five balanced datasets demonstrate that the Super-Metric is more stable and consistent than traditional metrics, exhibiting stronger correlations with the actual performance of classifiers. |
| title | Reducing Instability in Synthetic Data Evaluation with a Super-Metric in MalDataGen |
| topic | Artificial Intelligence Machine Learning 68T01 I.2 |
| url | https://arxiv.org/abs/2511.16373 |