Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.00408 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913292725780480 |
|---|---|
| author | Cruttwell, Geoffrey S. H. Gavranovic, Bruno Ghani, Neil Wilson, Paul Zanasi, Fabio |
| author_facet | Cruttwell, Geoffrey S. H. Gavranovic, Bruno Ghani, Neil Wilson, Paul Zanasi, Fabio |
| contents | We propose a categorical semantics for machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, and different architectures, shedding new light on their similarities and differences. Furthermore, our approach to learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realised in the discrete setting of Boolean and polynomial circuits. We demonstrate the practical significance of our framework with an implementation in Python. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_00408 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Deep Learning with Parametric Lenses Cruttwell, Geoffrey S. H. Gavranovic, Bruno Ghani, Neil Wilson, Paul Zanasi, Fabio Machine Learning Logic in Computer Science We propose a categorical semantics for machine learning algorithms in terms of lenses, parametric maps, and reverse derivative categories. This foundation provides a powerful explanatory and unifying framework: it encompasses a variety of gradient descent algorithms such as ADAM, AdaGrad, and Nesterov momentum, as well as a variety of loss functions such as MSE and Softmax cross-entropy, and different architectures, shedding new light on their similarities and differences. Furthermore, our approach to learning has examples generalising beyond the familiar continuous domains (modelled in categories of smooth maps) and can be realised in the discrete setting of Boolean and polynomial circuits. We demonstrate the practical significance of our framework with an implementation in Python. |
| title | Deep Learning with Parametric Lenses |
| topic | Machine Learning Logic in Computer Science |
| url | https://arxiv.org/abs/2404.00408 |