Convergence analysis of online algorithms for vector-valued kernel regression
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914070835232768 |
|---|---|
| author | Griebel, Michael Oswald, Peter |
| author_facet | Griebel, Michael Oswald, Peter |
| contents | We consider the problem of approximating the regression function $f_μ:\, Ω\to Y$ from noisy $μ$-distributed vector-valued data $(ω_m,y_m)\inΩ\times Y$ by an online learning algorithm using a reproducing kernel Hilbert space $H$ (RKHS) as prior. In an online algorithm, i.i.d. samples become available one by one via a random process and are successively processed to build approximations to the regression function. Assuming that the regression function essentially belongs to $H$ (soft learning scenario), we provide estimates for the expected squared error in the RKHS norm of the approximations $f^{(m)}\in H$ obtained by a standard regularized online approximation algorithm. In particular, we show an order-optimal estimate $$ \mathbb{E}(\|ε^{(m)}\|_H^2)\le C (m+1)^{-s/(2+s)},\qquad m=1,2,\ldots, $$ where $ε^{(m)}$ denotes the error term after $m$ processed data, the parameter $0<s\leq 1$ expresses an additional smoothness assumption on the regression function, and the constant $C$ depends on the variance of the input noise, the smoothness of the regression function, and other parameters of the algorithm. The proof, which is inspired by results on Schwarz iterative methods in the noiseless case, uses only elementary Hilbert space techniques and minimal assumptions on the noise, the feature map that defines $H$ and the associated covariance operator. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_07779 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Convergence analysis of online algorithms for vector-valued kernel regression Griebel, Michael Oswald, Peter Machine Learning Numerical Analysis 65D15, 65F08, 65F10, 68W27 We consider the problem of approximating the regression function $f_μ:\, Ω\to Y$ from noisy $μ$-distributed vector-valued data $(ω_m,y_m)\inΩ\times Y$ by an online learning algorithm using a reproducing kernel Hilbert space $H$ (RKHS) as prior. In an online algorithm, i.i.d. samples become available one by one via a random process and are successively processed to build approximations to the regression function. Assuming that the regression function essentially belongs to $H$ (soft learning scenario), we provide estimates for the expected squared error in the RKHS norm of the approximations $f^{(m)}\in H$ obtained by a standard regularized online approximation algorithm. In particular, we show an order-optimal estimate $$ \mathbb{E}(\|ε^{(m)}\|_H^2)\le C (m+1)^{-s/(2+s)},\qquad m=1,2,\ldots, $$ where $ε^{(m)}$ denotes the error term after $m$ processed data, the parameter $0<s\leq 1$ expresses an additional smoothness assumption on the regression function, and the constant $C$ depends on the variance of the input noise, the smoothness of the regression function, and other parameters of the algorithm. The proof, which is inspired by results on Schwarz iterative methods in the noiseless case, uses only elementary Hilbert space techniques and minimal assumptions on the noise, the feature map that defines $H$ and the associated covariance operator. |
| title | Convergence analysis of online algorithms for vector-valued kernel regression |
| topic | Machine Learning Numerical Analysis 65D15, 65F08, 65F10, 68W27 |
| url | https://arxiv.org/abs/2309.07779 |