Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.08986 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915489286979584 |
|---|---|
| author | Lian, Junbo Jacob |
| author_facet | Lian, Junbo Jacob |
| contents | Numerous purportedly improved metaheuristics claim superior performance based on equivalent function evaluations (FEs), yet often conceal additional computational burdens in more intensive iterations, preprocessing stages, or hyperparameter tuning. This paper posits that wall-clock time, rather than solely FEs, should serve as the principal budgetary constraint for equitable comparisons. We formalize a fixed-time, restart-fair benchmarking protocol wherein each algorithm is allotted an identical wall-clock time budget per problem instance, permitting unrestricted utilization of restarts, early termination criteria, and internal adaptive mechanisms. We advocate for the adoption of anytime performance curves, expected running time (ERT) metrics, and performance profiles that employ time as the cost measure, all aimed at predefined targets. Furthermore, we introduce a concise, reproducible checklist to standardize reporting practices and mitigate undisclosed computational overheads. This approach fosters more credible and practically relevant evaluations of metaheuristic algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_08986 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons Lian, Junbo Jacob Neural and Evolutionary Computing Performance Computation Numerous purportedly improved metaheuristics claim superior performance based on equivalent function evaluations (FEs), yet often conceal additional computational burdens in more intensive iterations, preprocessing stages, or hyperparameter tuning. This paper posits that wall-clock time, rather than solely FEs, should serve as the principal budgetary constraint for equitable comparisons. We formalize a fixed-time, restart-fair benchmarking protocol wherein each algorithm is allotted an identical wall-clock time budget per problem instance, permitting unrestricted utilization of restarts, early termination criteria, and internal adaptive mechanisms. We advocate for the adoption of anytime performance curves, expected running time (ERT) metrics, and performance profiles that employ time as the cost measure, all aimed at predefined targets. Furthermore, we introduce a concise, reproducible checklist to standardize reporting practices and mitigate undisclosed computational overheads. This approach fosters more credible and practically relevant evaluations of metaheuristic algorithms. |
| title | Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons |
| topic | Neural and Evolutionary Computing Performance Computation |
| url | https://arxiv.org/abs/2509.08986 |