Best free knot linear spline approximation and its application to neural networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913292115509248 |
|---|---|
| author | Peiris, Vinesha Pham, Duy Khoa Sukhorukova, Nadezda |
| author_facet | Peiris, Vinesha Pham, Duy Khoa Sukhorukova, Nadezda |
| contents | The problem of fixed knot approximation is convex and there are several efficient approaches to solve this problem, yet, when the knots joining the affine parts are also variable, finding conditions for a best Chebyshev approximation remains an open problem. It was noticed before that piecewise linear approximation with free knots is equivalent to neural network approximation with piecewise linear activation functions (for example ReLU). In this paper, we demonstrate that in the case of one internal free knot, the problem of linear spline approximation can be reformulated as a mixed-integer linear programming problem and solved efficiently using, for example, a branch and bound type method. We also present a new sufficient optimality condition for a one free knot piecewise linear approximation. The results of numerical experiments are provided. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_00008 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Best free knot linear spline approximation and its application to neural networks Peiris, Vinesha Pham, Duy Khoa Sukhorukova, Nadezda Optimization and Control The problem of fixed knot approximation is convex and there are several efficient approaches to solve this problem, yet, when the knots joining the affine parts are also variable, finding conditions for a best Chebyshev approximation remains an open problem. It was noticed before that piecewise linear approximation with free knots is equivalent to neural network approximation with piecewise linear activation functions (for example ReLU). In this paper, we demonstrate that in the case of one internal free knot, the problem of linear spline approximation can be reformulated as a mixed-integer linear programming problem and solved efficiently using, for example, a branch and bound type method. We also present a new sufficient optimality condition for a one free knot piecewise linear approximation. The results of numerical experiments are provided. |
| title | Best free knot linear spline approximation and its application to neural networks |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2404.00008 |