A note on the adjoint method for neural ordinary differential equation network
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910341769723904 |
|---|---|
| author | Hu, Pipi |
| author_facet | Hu, Pipi |
| contents | Perturbation and operator adjoint method are used to give the right adjoint form rigourously. From the derivation, we can have following results: 1) The loss gradient is not an ODE, it is an integral and we shows the reason; 2) The traditional adjoint form is not equivalent with the back propagation results. 3) The adjoint operator analysis shows that if and only if the discrete adjoint has the same scheme with the discrete neural ODE, the adjoint form would give the same results as BP does. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_15141 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A note on the adjoint method for neural ordinary differential equation network Hu, Pipi Numerical Analysis Machine Learning Perturbation and operator adjoint method are used to give the right adjoint form rigourously. From the derivation, we can have following results: 1) The loss gradient is not an ODE, it is an integral and we shows the reason; 2) The traditional adjoint form is not equivalent with the back propagation results. 3) The adjoint operator analysis shows that if and only if the discrete adjoint has the same scheme with the discrete neural ODE, the adjoint form would give the same results as BP does. |
| title | A note on the adjoint method for neural ordinary differential equation network |
| topic | Numerical Analysis Machine Learning |
| url | https://arxiv.org/abs/2402.15141 |