Implicit Bias of Mirror Flow on Separable Data
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916478812422144 |
|---|---|
| author | Pesme, Scott Dragomir, Radu-Alexandru Flammarion, Nicolas |
| author_facet | Pesme, Scott Dragomir, Radu-Alexandru Flammarion, Nicolas |
| contents | We examine the continuous-time counterpart of mirror descent, namely mirror flow, on classification problems which are linearly separable. Such problems are minimised `at infinity' and have many possible solutions; we study which solution is preferred by the algorithm depending on the mirror potential. For exponential tailed losses and under mild assumptions on the potential, we show that the iterates converge in direction towards a $ϕ_\infty$-maximum margin classifier. The function $ϕ_\infty$ is the \textit{horizon function} of the mirror potential and characterises its shape `at infinity'. When the potential is separable, a simple formula allows to compute this function. We analyse several examples of potentials and provide numerical experiments highlighting our results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_12763 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Implicit Bias of Mirror Flow on Separable Data Pesme, Scott Dragomir, Radu-Alexandru Flammarion, Nicolas Machine Learning Optimization and Control We examine the continuous-time counterpart of mirror descent, namely mirror flow, on classification problems which are linearly separable. Such problems are minimised `at infinity' and have many possible solutions; we study which solution is preferred by the algorithm depending on the mirror potential. For exponential tailed losses and under mild assumptions on the potential, we show that the iterates converge in direction towards a $ϕ_\infty$-maximum margin classifier. The function $ϕ_\infty$ is the \textit{horizon function} of the mirror potential and characterises its shape `at infinity'. When the potential is separable, a simple formula allows to compute this function. We analyse several examples of potentials and provide numerical experiments highlighting our results. |
| title | Implicit Bias of Mirror Flow on Separable Data |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2406.12763 |