Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.04512 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916471275257856 |
|---|---|
| author | Ebadulla, Danish Gulati, Aditya Singh, Ambuj |
| author_facet | Ebadulla, Danish Gulati, Aditya Singh, Ambuj |
| contents | We introduce a manifold analysis technique for neural network representations. Normalized Space Alignment (NSA) compares pairwise distances between two point clouds derived from the same source and having the same size, while potentially possessing differing dimensionalities. NSA can act as both an analytical tool and a differentiable loss function, providing a robust means of comparing and aligning representations across different layers and models. It satisfies the criteria necessary for both a similarity metric and a neural network loss function. We showcase NSA's versatility by illustrating its utility as a representation space analysis metric, a structure-preserving loss function, and a robustness analysis tool. NSA is not only computationally efficient but it can also approximate the global structural discrepancy during mini-batching, facilitating its use in a wide variety of neural network training paradigms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_04512 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Normalized Space Alignment: A Versatile Metric for Representation Analysis Ebadulla, Danish Gulati, Aditya Singh, Ambuj Machine Learning We introduce a manifold analysis technique for neural network representations. Normalized Space Alignment (NSA) compares pairwise distances between two point clouds derived from the same source and having the same size, while potentially possessing differing dimensionalities. NSA can act as both an analytical tool and a differentiable loss function, providing a robust means of comparing and aligning representations across different layers and models. It satisfies the criteria necessary for both a similarity metric and a neural network loss function. We showcase NSA's versatility by illustrating its utility as a representation space analysis metric, a structure-preserving loss function, and a robustness analysis tool. NSA is not only computationally efficient but it can also approximate the global structural discrepancy during mini-batching, facilitating its use in a wide variety of neural network training paradigms. |
| title | Normalized Space Alignment: A Versatile Metric for Representation Analysis |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2411.04512 |