Defining Neural Network Architecture through Polytope Structures of Dataset
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916265514237952 |
|---|---|
| author | Lee, Sangmin Mammadov, Abbas Ye, Jong Chul |
| author_facet | Lee, Sangmin Mammadov, Abbas Ye, Jong Chul |
| contents | Current theoretical and empirical research in neural networks suggests that complex datasets require large network architectures for thorough classification, yet the precise nature of this relationship remains unclear. This paper tackles this issue by defining upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We also delve into the application of these principles to simplicial complexes and specific manifold shapes, explaining how the requirement for network width varies in accordance with the geometric complexity of the dataset. Moreover, we develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. Through our algorithm, it is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02407 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Defining Neural Network Architecture through Polytope Structures of Dataset Lee, Sangmin Mammadov, Abbas Ye, Jong Chul Machine Learning Computer Vision and Pattern Recognition Neural and Evolutionary Computing Current theoretical and empirical research in neural networks suggests that complex datasets require large network architectures for thorough classification, yet the precise nature of this relationship remains unclear. This paper tackles this issue by defining upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We also delve into the application of these principles to simplicial complexes and specific manifold shapes, explaining how the requirement for network width varies in accordance with the geometric complexity of the dataset. Moreover, we develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. Through our algorithm, it is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces. |
| title | Defining Neural Network Architecture through Polytope Structures of Dataset |
| topic | Machine Learning Computer Vision and Pattern Recognition Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2402.02407 |