Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911735262216192 |
|---|---|
| author | Wang, Jialiang Liu, Hanmo Di, Shimin Wang, Zhili Wang, Jiachuan Chen, Lei Zhou, Xiaofang |
| author_facet | Wang, Jialiang Liu, Hanmo Di, Shimin Wang, Zhili Wang, Jiachuan Chen, Lei Zhou, Xiaofang |
| contents | Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15336 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design Wang, Jialiang Liu, Hanmo Di, Shimin Wang, Zhili Wang, Jiachuan Chen, Lei Zhou, Xiaofang Machine Learning Artificial Intelligence Databases Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency. Current methods fail to achieve this balance: neural architectural search is computationally expensive, while model retrieval often yields suboptimal static checkpoints. To resolve this dilemma, we model the performance gains induced by fine-grained architectural modifications as edit-effect evidence and build evidence graphs from prior tasks. By constructing a retrieval-augmented model refinement framework, our proposed M-DESIGN dynamically weaves historical evidence to discover near-optimal modification paths. M-DESIGN features an adaptive retrieval mechanism that quickly calibrates the evolving transferability of edit-effect evidence from different sources. To handle out-of-distribution shifts, we introduce predictive task planners that extrapolate gains from multi-hop evidence, thereby reducing reliance on an exhaustive repository. Based on our model knowledge base of 67,760 graph neural networks across 22 datasets, extensive experiments demonstrate that M-DESIGN consistently outperforms baselines, achieving the search-space best performance in 26 out of 33 cases under a strict budget. |
| title | Beyond Model Base Retrieval: Weaving Knowledge to Master Fine-grained Neural Network Design |
| topic | Machine Learning Artificial Intelligence Databases |
| url | https://arxiv.org/abs/2507.15336 |