Towards Efficient Few-shot Graph Neural Architecture Search via Partitioning Gradient Contribution

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Wenhao, Wu, Xuan, Yang, Bo, Zhou, You, Xiao, Yubin, Liang, Yanchun, Ge, Hongwei, Lee, Heow Pueh, Wu, Chunguo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866909653694152704
author	Song, Wenhao Wu, Xuan Yang, Bo Zhou, You Xiao, Yubin Liang, Yanchun Ge, Hongwei Lee, Heow Pueh Wu, Chunguo
author_facet	Song, Wenhao Wu, Xuan Yang, Bo Zhou, You Xiao, Yubin Liang, Yanchun Ge, Hongwei Lee, Heow Pueh Wu, Chunguo
contents	To address the weight coupling problem, certain studies introduced few-shot Neural Architecture Search (NAS) methods, which partition the supernet into multiple sub-supernets. However, these methods often suffer from computational inefficiency and tend to provide suboptimal partitioning schemes. To address this problem more effectively, we analyze the weight coupling problem from a novel perspective, which primarily stems from distinct modules in succeeding layers imposing conflicting gradient directions on the preceding layer modules. Based on this perspective, we propose the Gradient Contribution (GC) method that efficiently computes the cosine similarity of gradient directions among modules by decomposing the Vector-Jacobian Product during supernet backpropagation. Subsequently, the modules with conflicting gradient directions are allocated to distinct sub-supernets while similar ones are grouped together. To assess the advantages of GC and address the limitations of existing Graph Neural Architecture Search methods, which are limited to searching a single type of Graph Neural Networks (Message Passing Neural Networks (MPNNs) or Graph Transformers (GTs)), we propose the Unified Graph Neural Architecture Search (UGAS) framework, which explores optimal combinations of MPNNs and GTs. The experimental results demonstrate that GC achieves state-of-the-art (SOTA) performance in supernet partitioning quality and time efficiency. In addition, the architectures searched by UGAS+GC outperform both the manually designed GNNs and those obtained by existing NAS methods. Finally, ablation studies further demonstrate the effectiveness of all proposed methods.
format	Preprint
id	arxiv_https___arxiv_org_abs_2506_01231
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Towards Efficient Few-shot Graph Neural Architecture Search via Partitioning Gradient Contribution Song, Wenhao Wu, Xuan Yang, Bo Zhou, You Xiao, Yubin Liang, Yanchun Ge, Hongwei Lee, Heow Pueh Wu, Chunguo Machine Learning Artificial Intelligence To address the weight coupling problem, certain studies introduced few-shot Neural Architecture Search (NAS) methods, which partition the supernet into multiple sub-supernets. However, these methods often suffer from computational inefficiency and tend to provide suboptimal partitioning schemes. To address this problem more effectively, we analyze the weight coupling problem from a novel perspective, which primarily stems from distinct modules in succeeding layers imposing conflicting gradient directions on the preceding layer modules. Based on this perspective, we propose the Gradient Contribution (GC) method that efficiently computes the cosine similarity of gradient directions among modules by decomposing the Vector-Jacobian Product during supernet backpropagation. Subsequently, the modules with conflicting gradient directions are allocated to distinct sub-supernets while similar ones are grouped together. To assess the advantages of GC and address the limitations of existing Graph Neural Architecture Search methods, which are limited to searching a single type of Graph Neural Networks (Message Passing Neural Networks (MPNNs) or Graph Transformers (GTs)), we propose the Unified Graph Neural Architecture Search (UGAS) framework, which explores optimal combinations of MPNNs and GTs. The experimental results demonstrate that GC achieves state-of-the-art (SOTA) performance in supernet partitioning quality and time efficiency. In addition, the architectures searched by UGAS+GC outperform both the manually designed GNNs and those obtained by existing NAS methods. Finally, ablation studies further demonstrate the effectiveness of all proposed methods.
title	Towards Efficient Few-shot Graph Neural Architecture Search via Partitioning Gradient Contribution
topic	Machine Learning Artificial Intelligence
url	https://arxiv.org/abs/2506.01231