Spectral Architecture Search for Neural Network Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Peri, Gianluca, Chicchi, Lorenzo, Fanelli, Duccio, Giambagli, Lorenzo
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866912520379301888
author	Peri, Gianluca Chicchi, Lorenzo Fanelli, Duccio Giambagli, Lorenzo
author_facet	Peri, Gianluca Chicchi, Lorenzo Fanelli, Duccio Giambagli, Lorenzo
contents	Architecture design and optimization are challenging problems in the field of artificial neural networks. Working in this context, we here present SPARCS (SPectral ARchiteCture Search), a novel architecture search protocol which exploits the spectral attributes of the inter-layer transfer matrices. SPARCS allows one to explore the space of possible architectures by spanning continuous and differentiable manifolds, thus enabling for gradient-based optimization algorithms to be eventually employed. With reference to simple benchmark models, we show that the newly proposed method yields a self-emerging architecture with a minimal degree of expressivity to handle the task under investigation and with a reduced parameter count as compared to other viable alternatives.
format	Preprint
id	arxiv_https___arxiv_org_abs_2504_00885
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Spectral Architecture Search for Neural Network Models Peri, Gianluca Chicchi, Lorenzo Fanelli, Duccio Giambagli, Lorenzo Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence Architecture design and optimization are challenging problems in the field of artificial neural networks. Working in this context, we here present SPARCS (SPectral ARchiteCture Search), a novel architecture search protocol which exploits the spectral attributes of the inter-layer transfer matrices. SPARCS allows one to explore the space of possible architectures by spanning continuous and differentiable manifolds, thus enabling for gradient-based optimization algorithms to be eventually employed. With reference to simple benchmark models, we show that the newly proposed method yields a self-emerging architecture with a minimal degree of expressivity to handle the task under investigation and with a reduced parameter count as compared to other viable alternatives.
title	Spectral Architecture Search for Neural Network Models
topic	Machine Learning Disordered Systems and Neural Networks Statistical Mechanics Artificial Intelligence
url	https://arxiv.org/abs/2504.00885