Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2405.13972 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929573068800000 |
|---|---|
| author | Xu, Chenhui Yu, Fuxun Li, Maoliang Zheng, Zihao Xu, Zirui Xiong, Jinjun Chen, Xiang |
| author_facet | Xu, Chenhui Yu, Fuxun Li, Maoliang Zheng, Zihao Xu, Zirui Xiong, Jinjun Chen, Xiang |
| contents | The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling.
Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_13972 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Infinite-Dimensional Feature Interaction Xu, Chenhui Yu, Fuxun Li, Maoliang Zheng, Zihao Xu, Zirui Xiong, Jinjun Chen, Xiang Machine Learning The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance. |
| title | Infinite-Dimensional Feature Interaction |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2405.13972 |