MOLE: MOdular Learning FramEwork via Mutual Information Maximization
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913167205990400 |
|---|---|
| author | Li, Tianchao Pei, Yulong |
| author_facet | Li, Tianchao Pei, Yulong |
| contents | This paper is to introduce an asynchronous and local learning framework for neural networks, named Modular Learning Framework (MOLE). This framework modularizes neural networks by layers, defines the training objective via mutual information for each module, and sequentially trains each module by mutual information maximization. MOLE makes the training become local optimization with gradient-isolated across modules, and this scheme is more biologically plausible than BP. We run experiments on vector-, grid- and graph-type data. In particular, this framework is capable of solving both graph- and node-level tasks for graph-type data. Therefore, MOLE has been experimentally proven to be universally applicable to different types of data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_07772 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | MOLE: MOdular Learning FramEwork via Mutual Information Maximization Li, Tianchao Pei, Yulong Machine Learning Artificial Intelligence This paper is to introduce an asynchronous and local learning framework for neural networks, named Modular Learning Framework (MOLE). This framework modularizes neural networks by layers, defines the training objective via mutual information for each module, and sequentially trains each module by mutual information maximization. MOLE makes the training become local optimization with gradient-isolated across modules, and this scheme is more biologically plausible than BP. We run experiments on vector-, grid- and graph-type data. In particular, this framework is capable of solving both graph- and node-level tasks for graph-type data. Therefore, MOLE has been experimentally proven to be universally applicable to different types of data. |
| title | MOLE: MOdular Learning FramEwork via Mutual Information Maximization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2308.07772 |