Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916653255622656 |
|---|---|
| author | Duan, Shijin Liu, Yejia Liu, Gaowen Kompella, Ramana Rao Ren, Shaolei Xu, Xiaolin |
| author_facet | Duan, Shijin Liu, Yejia Liu, Gaowen Kompella, Ramana Rao Ren, Shaolei Xu, Xiaolin |
| contents | Vector Symbolic Architecture (VSA) is emerging in machine learning due to its efficiency, but they are hindered by issues of hyperdimensionality and accuracy. As a promising mitigation, the Low-Dimensional Computing (LDC) method significantly reduces the vector dimension by ~100 times while maintaining accuracy, by employing a gradient-based optimization. Despite its potential, LDC optimization for VSA is still underexplored. Our investigation into vector updates underscores the importance of stable, adaptive dynamics in LDC training. We also reveal the overlooked yet critical roles of batch normalization (BN) and knowledge distillation (KD) in standard approaches. Besides the accuracy boost, BN does not add computational overhead during inference, and KD significantly enhances inference confidence. Through extensive experiments and ablation studies across multiple benchmarks, we provide a thorough evaluation of our approach and extend the interpretability of binary neural network optimization similar to LDC, previously unaddressed in BNN literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_14075 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture Duan, Shijin Liu, Yejia Liu, Gaowen Kompella, Ramana Rao Ren, Shaolei Xu, Xiaolin Machine Learning Vector Symbolic Architecture (VSA) is emerging in machine learning due to its efficiency, but they are hindered by issues of hyperdimensionality and accuracy. As a promising mitigation, the Low-Dimensional Computing (LDC) method significantly reduces the vector dimension by ~100 times while maintaining accuracy, by employing a gradient-based optimization. Despite its potential, LDC optimization for VSA is still underexplored. Our investigation into vector updates underscores the importance of stable, adaptive dynamics in LDC training. We also reveal the overlooked yet critical roles of batch normalization (BN) and knowledge distillation (KD) in standard approaches. Besides the accuracy boost, BN does not add computational overhead during inference, and KD significantly enhances inference confidence. Through extensive experiments and ablation studies across multiple benchmarks, we provide a thorough evaluation of our approach and extend the interpretability of binary neural network optimization similar to LDC, previously unaddressed in BNN literature. |
| title | Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.14075 |