Kolmogorov-Arnold Fourier Networks
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911711616827392 |
|---|---|
| author | Zhang, Jusheng Fan, Yijia Cai, Kaitong Wang, Keze Wang, Wenhao |
| author_facet | Zhang, Jusheng Fan, Yijia Cai, Kaitong Wang, Keze Wang, Wenhao |
| contents | Although Kolmogorov-Arnold-based interpretable networks (KANs) possess strong theoretical expressiveness, they suffer from severe parameter explosion and limited ability to capture high-frequency features in high-dimensional tasks. To address these issues, we propose the Kolmogorov-Arnold Fourier Network (KAF), which fundamentally redefines the KAN paradigm through spectral reparameterization. Our key contributions include: (1) proposing a fundamental basis transformation from the local, grid-based B-spline representation to a global, adaptive spectral representation. This shift changes the network's inductive bias, reducing parameter complexity from $O(G)$ to $O(1)$ while preserving expressiveness; (2) introducing trainable Random Fourier Features (RFF) initialized via a spectral alignment strategy, which allows the model to break the smoothness limitation of fixed kernels and accurately capture high-frequency components; and (3) implementing an adaptive hybrid GELU-Fourier activation mechanism that progressively enhances frequency representation during training. Comprehensive experiments demonstrate the superiority of KAF across computer vision (CV), natural language processing (NLP), audio, and partial differential equation (PDE) solving tasks, achieving state-of-the-art performance with improved efficiency. The code is available at https://github.com/kolmogorovArnoldFourierNetwork/KAF. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_06018 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Kolmogorov-Arnold Fourier Networks Zhang, Jusheng Fan, Yijia Cai, Kaitong Wang, Keze Wang, Wenhao Machine Learning Artificial Intelligence Although Kolmogorov-Arnold-based interpretable networks (KANs) possess strong theoretical expressiveness, they suffer from severe parameter explosion and limited ability to capture high-frequency features in high-dimensional tasks. To address these issues, we propose the Kolmogorov-Arnold Fourier Network (KAF), which fundamentally redefines the KAN paradigm through spectral reparameterization. Our key contributions include: (1) proposing a fundamental basis transformation from the local, grid-based B-spline representation to a global, adaptive spectral representation. This shift changes the network's inductive bias, reducing parameter complexity from $O(G)$ to $O(1)$ while preserving expressiveness; (2) introducing trainable Random Fourier Features (RFF) initialized via a spectral alignment strategy, which allows the model to break the smoothness limitation of fixed kernels and accurately capture high-frequency components; and (3) implementing an adaptive hybrid GELU-Fourier activation mechanism that progressively enhances frequency representation during training. Comprehensive experiments demonstrate the superiority of KAF across computer vision (CV), natural language processing (NLP), audio, and partial differential equation (PDE) solving tasks, achieving state-of-the-art performance with improved efficiency. The code is available at https://github.com/kolmogorovArnoldFourierNetwork/KAF. |
| title | Kolmogorov-Arnold Fourier Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.06018 |