Mean-field limit from general mixtures of experts to quantum neural networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917451584765952 |
|---|---|
| author | Hernandez, Anderson Melchor Pastorello, Davide De Palma, Giacomo |
| author_facet | Hernandez, Anderson Melchor Pastorello, Davide De Palma, Giacomo |
| contents | In this work, we study the asymptotic behavior of Mixture of Experts (MoE) trained via gradient flow on supervised learning problems. Our main result establishes the propagation of chaos for a MoE as the number of experts diverges. We demonstrate that the corresponding empirical measure of their parameters is close to a probability measure that solves a nonlinear continuity equation, and we provide an explicit convergence rate that depends solely on the number of experts. We apply our results to a MoE generated by a quantum neural network. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14660 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mean-field limit from general mixtures of experts to quantum neural networks Hernandez, Anderson Melchor Pastorello, Davide De Palma, Giacomo Mathematical Physics Machine Learning Probability 81P45, 49Q22, 60F05 In this work, we study the asymptotic behavior of Mixture of Experts (MoE) trained via gradient flow on supervised learning problems. Our main result establishes the propagation of chaos for a MoE as the number of experts diverges. We demonstrate that the corresponding empirical measure of their parameters is close to a probability measure that solves a nonlinear continuity equation, and we provide an explicit convergence rate that depends solely on the number of experts. We apply our results to a MoE generated by a quantum neural network. |
| title | Mean-field limit from general mixtures of experts to quantum neural networks |
| topic | Mathematical Physics Machine Learning Probability 81P45, 49Q22, 60F05 |
| url | https://arxiv.org/abs/2501.14660 |