Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2018
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/1802.06760 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918286281670656 |
|---|---|
| author | Karatapanis, Konstantinos |
| author_facet | Karatapanis, Konstantinos |
| contents | We consider SDEs of the form $dX_t = |f(X_t)|/t^γ dt+1/t^γ dB_t$, where $f(x)$ behaves comparably to $|x|^k$ in a neighborhood of the origin, for $k\in [1,\infty)$. We show that there exists a threshold value $:=\tildeγ$ for $γ$, depending on $k$, such that when $γ\in (1/2, \tildeγ)$ then $\mathbb{P}(X_n\rightarrow 0) = 0$, and for the rest of the permissible values $\mathbb{P}(X_n\rightarrow 0)>0$. The previous results extend for discrete processes that satisfy $X_{n+1}-X_n = f(X_n)/n^γ+Y_n/n^γ$. Here, $Y_{n+1}$ are martingale differences that are a.s. bounded.
This result shows that for a function $F$, whose second derivative at degenerate saddle points is of polynomial order, it is always possible to escape saddle points via the iteration $X_{n+1}-X_n =F'(X_n)/n^γ+Y_n/n^γ$ for a suitable choice of $γ$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1802_06760 |
| institution | arXiv |
| publishDate | 2018 |
| record_format | arxiv |
| spellingShingle | One-dimensional System Arising in Stochastic Gradient Descent Karatapanis, Konstantinos Probability We consider SDEs of the form $dX_t = |f(X_t)|/t^γ dt+1/t^γ dB_t$, where $f(x)$ behaves comparably to $|x|^k$ in a neighborhood of the origin, for $k\in [1,\infty)$. We show that there exists a threshold value $:=\tildeγ$ for $γ$, depending on $k$, such that when $γ\in (1/2, \tildeγ)$ then $\mathbb{P}(X_n\rightarrow 0) = 0$, and for the rest of the permissible values $\mathbb{P}(X_n\rightarrow 0)>0$. The previous results extend for discrete processes that satisfy $X_{n+1}-X_n = f(X_n)/n^γ+Y_n/n^γ$. Here, $Y_{n+1}$ are martingale differences that are a.s. bounded. This result shows that for a function $F$, whose second derivative at degenerate saddle points is of polynomial order, it is always possible to escape saddle points via the iteration $X_{n+1}-X_n =F'(X_n)/n^γ+Y_n/n^γ$ for a suitable choice of $γ$. |
| title | One-dimensional System Arising in Stochastic Gradient Descent |
| topic | Probability |
| url | https://arxiv.org/abs/1802.06760 |