Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2407.15580 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912191388581888 |
|---|---|
| author | Perera, David Letzelter, Victor Mariotte, Théo Cortés, Adrien Chen, Mickael Essid, Slim Richard, Gaël |
| author_facet | Perera, David Letzelter, Victor Mariotte, Théo Cortés, Adrien Chen, Mickael Essid, Slim Richard, Gaël |
| contents | We introduce Annealed Multiple Choice Learning (aMCL) which combines simulated annealing with MCL. MCL is a learning framework handling ambiguous tasks by predicting a small set of plausible hypotheses. These hypotheses are trained using the Winner-takes-all (WTA) scheme, which promotes the diversity of the predictions. However, this scheme may converge toward an arbitrarily suboptimal local minimum, due to the greedy nature of WTA. We overcome this limitation using annealing, which enhances the exploration of the hypothesis space during training. We leverage insights from statistical physics and information theory to provide a detailed description of the model training trajectory. Additionally, we validate our algorithm by extensive experiments on synthetic datasets, on the standard UCI benchmark, and on speech separation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_15580 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing Perera, David Letzelter, Victor Mariotte, Théo Cortés, Adrien Chen, Mickael Essid, Slim Richard, Gaël Machine Learning Sound Audio and Speech Processing Probability We introduce Annealed Multiple Choice Learning (aMCL) which combines simulated annealing with MCL. MCL is a learning framework handling ambiguous tasks by predicting a small set of plausible hypotheses. These hypotheses are trained using the Winner-takes-all (WTA) scheme, which promotes the diversity of the predictions. However, this scheme may converge toward an arbitrarily suboptimal local minimum, due to the greedy nature of WTA. We overcome this limitation using annealing, which enhances the exploration of the hypothesis space during training. We leverage insights from statistical physics and information theory to provide a detailed description of the model training trajectory. Additionally, we validate our algorithm by extensive experiments on synthetic datasets, on the standard UCI benchmark, and on speech separation. |
| title | Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing |
| topic | Machine Learning Sound Audio and Speech Processing Probability |
| url | https://arxiv.org/abs/2407.15580 |