RedEx: Beyond Fixed Representation Methods via Convex Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911758367588352 |
|---|---|
| author | Daniely, Amit Schain, Mariano Yehudai, Gilad |
| author_facet | Daniely, Amit Schain, Mariano Yehudai, Gilad |
| contents | Optimizing Neural networks is a difficult task which is still not well understood. On the other hand, fixed representation methods such as kernels and random features have provable optimization guarantees but inferior performance due to their inherent inability to learn the representations. In this paper, we aim at bridging this gap by presenting a novel architecture called RedEx (Reduced Expander Extractor) that is as expressive as neural networks and can also be trained in a layer-wise fashion via a convex program with semi-definite constraints and optimization guarantees. We also show that RedEx provably surpasses fixed representation methods, in the sense that it can efficiently learn a family of target functions which fixed representation methods cannot. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_07606 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | RedEx: Beyond Fixed Representation Methods via Convex Optimization Daniely, Amit Schain, Mariano Yehudai, Gilad Machine Learning Optimization and Control Optimizing Neural networks is a difficult task which is still not well understood. On the other hand, fixed representation methods such as kernels and random features have provable optimization guarantees but inferior performance due to their inherent inability to learn the representations. In this paper, we aim at bridging this gap by presenting a novel architecture called RedEx (Reduced Expander Extractor) that is as expressive as neural networks and can also be trained in a layer-wise fashion via a convex program with semi-definite constraints and optimization guarantees. We also show that RedEx provably surpasses fixed representation methods, in the sense that it can efficiently learn a family of target functions which fixed representation methods cannot. |
| title | RedEx: Beyond Fixed Representation Methods via Convex Optimization |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2401.07606 |