DiagrammaticLearning: A Graphical Language for Compositional Training Regimes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929657007308800 |
|---|---|
| author | Lary, Mason Samuelson, Richard Wilentz, Alexander Zare, Alina Klawonn, Matthew Fairbanks, James P. |
| author_facet | Lary, Mason Samuelson, Richard Wilentz, Alexander Zare, Alina Klawonn, Matthew Fairbanks, James P. |
| contents | Motivated by deep learning regimes with multiple interacting yet distinct model components, we introduce learning diagrams, graphical depictions of training setups that capture parameterized learning as data rather than code. A learning diagram compiles to a unique loss function on which component models are trained. The result of training on this loss is a collection of models whose predictions ``agree" with one another. We show that a number of popular learning setups such as few-shot multi-task learning, knowledge distillation, and multi-modal learning can be depicted as learning diagrams. We further implement learning diagrams in a library that allows users to build diagrams of PyTorch and Flux.jl models. By implementing some classic machine learning use cases, we demonstrate how learning diagrams allow practitioners to build complicated models as compositions of smaller components, identify relationships between workflows, and manipulate models during or after training. Leveraging a category theoretic framework, we introduce a rigorous semantics for learning diagrams that puts such operations on a firm mathematical foundation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_01515 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DiagrammaticLearning: A Graphical Language for Compositional Training Regimes Lary, Mason Samuelson, Richard Wilentz, Alexander Zare, Alina Klawonn, Matthew Fairbanks, James P. Machine Learning Artificial Intelligence Programming Languages Category Theory Motivated by deep learning regimes with multiple interacting yet distinct model components, we introduce learning diagrams, graphical depictions of training setups that capture parameterized learning as data rather than code. A learning diagram compiles to a unique loss function on which component models are trained. The result of training on this loss is a collection of models whose predictions ``agree" with one another. We show that a number of popular learning setups such as few-shot multi-task learning, knowledge distillation, and multi-modal learning can be depicted as learning diagrams. We further implement learning diagrams in a library that allows users to build diagrams of PyTorch and Flux.jl models. By implementing some classic machine learning use cases, we demonstrate how learning diagrams allow practitioners to build complicated models as compositions of smaller components, identify relationships between workflows, and manipulate models during or after training. Leveraging a category theoretic framework, we introduce a rigorous semantics for learning diagrams that puts such operations on a firm mathematical foundation. |
| title | DiagrammaticLearning: A Graphical Language for Compositional Training Regimes |
| topic | Machine Learning Artificial Intelligence Programming Languages Category Theory |
| url | https://arxiv.org/abs/2501.01515 |