VectorSearch: Enhancing Document Retrieval with Semantic Embeddings and Optimized Search
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917787363966976 |
|---|---|
| author | Monir, Solmaz Seyed Lau, Irene Yang, Shubing Zhao, Dongfang |
| author_facet | Monir, Solmaz Seyed Lau, Irene Yang, Shubing Zhao, Dongfang |
| contents | Traditional retrieval methods have been essential for assessing document similarity but struggle with capturing semantic nuances. Despite advancements in latent semantic analysis (LSA) and deep learning, achieving comprehensive semantic understanding and accurate retrieval remains challenging due to high dimensionality and semantic gaps. The above challenges call for new techniques to effectively reduce the dimensions and close the semantic gaps. To this end, we propose VectorSearch, which leverages advanced algorithms, embeddings, and indexing techniques for refined retrieval. By utilizing innovative multi-vector search operations and encoding searches with advanced language models, our approach significantly improves retrieval accuracy. Experiments on real-world datasets show that VectorSearch outperforms baseline metrics, demonstrating its efficacy for large-scale retrieval tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_17383 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VectorSearch: Enhancing Document Retrieval with Semantic Embeddings and Optimized Search Monir, Solmaz Seyed Lau, Irene Yang, Shubing Zhao, Dongfang Information Retrieval Artificial Intelligence Databases Machine Learning Performance Traditional retrieval methods have been essential for assessing document similarity but struggle with capturing semantic nuances. Despite advancements in latent semantic analysis (LSA) and deep learning, achieving comprehensive semantic understanding and accurate retrieval remains challenging due to high dimensionality and semantic gaps. The above challenges call for new techniques to effectively reduce the dimensions and close the semantic gaps. To this end, we propose VectorSearch, which leverages advanced algorithms, embeddings, and indexing techniques for refined retrieval. By utilizing innovative multi-vector search operations and encoding searches with advanced language models, our approach significantly improves retrieval accuracy. Experiments on real-world datasets show that VectorSearch outperforms baseline metrics, demonstrating its efficacy for large-scale retrieval tasks. |
| title | VectorSearch: Enhancing Document Retrieval with Semantic Embeddings and Optimized Search |
| topic | Information Retrieval Artificial Intelligence Databases Machine Learning Performance |
| url | https://arxiv.org/abs/2409.17383 |