Cost-Effective Model Evaluation with Meta-Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910248049049600 |
|---|---|
| author | Pham, Trinh Huynh, Viet Yin, Hongzhi Nguyen, Quoc Viet Hung Nguyen, Thanh Tam |
| author_facet | Pham, Trinh Huynh, Viet Yin, Hongzhi Nguyen, Quoc Viet Hung Nguyen, Thanh Tam |
| contents | The rapid growth of machine learning has produced an ever-expanding ecosystem of models, making it increasingly challenging to verify the reliability of newly released models on unseen, unlabeled data. Conventional evaluation pipelines depend on expensive annotation, repeated fine-tuning, or narrow assumptions that fail to transfer across model families. We present MetaEvaluator, a cost-effective, model-agnostic framework for rapid, label-free assessment of unseen models spanning diverse architectures and modalities. MetaEvaluator leverages meta-learning over a pool of reference models to obtain a transferable initialization, enabling accurate evaluation of new models while amortizing cost across the pool and removing the need for per-model retraining. To the best of our knowledge, this is the first model-agnostic framework capable of evaluating new models on entirely unlabeled datasets. Extensive experiments show that MetaEvaluator produces stable and accurate performance estimates at substantially reduced cost compared to conventional approaches, making scalable benchmarking of emerging models on unlabeled data practical. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_23595 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Cost-Effective Model Evaluation with Meta-Learning Pham, Trinh Huynh, Viet Yin, Hongzhi Nguyen, Quoc Viet Hung Nguyen, Thanh Tam Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Emerging Technologies Performance The rapid growth of machine learning has produced an ever-expanding ecosystem of models, making it increasingly challenging to verify the reliability of newly released models on unseen, unlabeled data. Conventional evaluation pipelines depend on expensive annotation, repeated fine-tuning, or narrow assumptions that fail to transfer across model families. We present MetaEvaluator, a cost-effective, model-agnostic framework for rapid, label-free assessment of unseen models spanning diverse architectures and modalities. MetaEvaluator leverages meta-learning over a pool of reference models to obtain a transferable initialization, enabling accurate evaluation of new models while amortizing cost across the pool and removing the need for per-model retraining. To the best of our knowledge, this is the first model-agnostic framework capable of evaluating new models on entirely unlabeled datasets. Extensive experiments show that MetaEvaluator produces stable and accurate performance estimates at substantially reduced cost compared to conventional approaches, making scalable benchmarking of emerging models on unlabeled data practical. |
| title | Cost-Effective Model Evaluation with Meta-Learning |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Emerging Technologies Performance |
| url | https://arxiv.org/abs/2605.23595 |