Neural Networks Learn Distance Metrics
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909475744514048 |
|---|---|
| author | Oursland, Alan |
| author_facet | Oursland, Alan |
| contents | Neural networks may naturally favor distance-based representations, where smaller activations indicate closer proximity to learned prototypes. This contrasts with intensity-based approaches, which rely on activation magnitudes. To test this hypothesis, we conducted experiments with six MNIST architectural variants constrained to learn either distance or intensity representations. Our results reveal that the underlying representation affects model performance. We develop a novel geometric framework that explains these findings and introduce OffsetL2, a new architecture based on Mahalanobis distance equations, to further validate this framework. This work highlights the importance of considering distance-based learning in neural network design. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02103 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Neural Networks Learn Distance Metrics Oursland, Alan Machine Learning Artificial Intelligence 68T07 (Primary) 62H12 (Secondary) I.5.1; G.3 Neural networks may naturally favor distance-based representations, where smaller activations indicate closer proximity to learned prototypes. This contrasts with intensity-based approaches, which rely on activation magnitudes. To test this hypothesis, we conducted experiments with six MNIST architectural variants constrained to learn either distance or intensity representations. Our results reveal that the underlying representation affects model performance. We develop a novel geometric framework that explains these findings and introduce OffsetL2, a new architecture based on Mahalanobis distance equations, to further validate this framework. This work highlights the importance of considering distance-based learning in neural network design. |
| title | Neural Networks Learn Distance Metrics |
| topic | Machine Learning Artificial Intelligence 68T07 (Primary) 62H12 (Secondary) I.5.1; G.3 |
| url | https://arxiv.org/abs/2502.02103 |