EnterpriseEM: Fine-tuned Embeddings for Enterprise Semantic Search
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914180536205312 |
|---|---|
| author | Rathinasamy, Kamalkumar Nettar, Jayarama Kumar, Amit Manchanda, Vishal Vijayakumar, Arun Kataria, Ayush Manjunath, Venkateshprasanna GS, Chidambaram Sodhi, Jaskirat Singh Shaikh, Shoeb Khan, Wasim Akhtar Singh, Prashant Ige, Tanishq Dattatray Tiwari, Vipin Mondal, Rajab Ali K, Harshini Reka, S Amancharla, Chetana Rahman, Faiz ur A, Harikrishnan P Saha, Indraneel Tiwary, Bhavya Patel, Navin Shankar S, Pradeep T J, Balaji A Priyapravas Tarafdar, Mohammed Rafee |
| author_facet | Rathinasamy, Kamalkumar Nettar, Jayarama Kumar, Amit Manchanda, Vishal Vijayakumar, Arun Kataria, Ayush Manjunath, Venkateshprasanna GS, Chidambaram Sodhi, Jaskirat Singh Shaikh, Shoeb Khan, Wasim Akhtar Singh, Prashant Ige, Tanishq Dattatray Tiwari, Vipin Mondal, Rajab Ali K, Harshini Reka, S Amancharla, Chetana Rahman, Faiz ur A, Harikrishnan P Saha, Indraneel Tiwary, Bhavya Patel, Navin Shankar S, Pradeep T J, Balaji A Priyapravas Tarafdar, Mohammed Rafee |
| contents | Enterprises grapple with the significant challenge of managing proprietary unstructured data, hindering efficient information retrieval. This has led to the emergence of AI-driven information retrieval solutions, designed to adeptly extract relevant insights to address employee inquiries. These solutions often leverage pre-trained embedding models and generative models as foundational components. While pre-trained embeddings may exhibit proximity or disparity based on their original training objectives, they might not fully align with the unique characteristics of enterprise-specific data, leading to suboptimal alignment with the retrieval goals of enterprise environments. In this paper, we propose a comprehensive methodology for contextualizing pre-trained embedding models to enterprise environments, covering the entire process from data preparation to model fine-tuning and evaluation. By adapting the embeddings to better suit the retrieval tasks prevalent in enterprises, we aim to enhance the performance of information retrieval solutions. We discuss the process of fine-tuning, its effect on retrieval accuracy, and the potential benefits for enterprise information management. Our findings demonstrate the efficacy of fine-tuned embedding models in improving the precision and relevance of search results in enterprise settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00010 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EnterpriseEM: Fine-tuned Embeddings for Enterprise Semantic Search Rathinasamy, Kamalkumar Nettar, Jayarama Kumar, Amit Manchanda, Vishal Vijayakumar, Arun Kataria, Ayush Manjunath, Venkateshprasanna GS, Chidambaram Sodhi, Jaskirat Singh Shaikh, Shoeb Khan, Wasim Akhtar Singh, Prashant Ige, Tanishq Dattatray Tiwari, Vipin Mondal, Rajab Ali K, Harshini Reka, S Amancharla, Chetana Rahman, Faiz ur A, Harikrishnan P Saha, Indraneel Tiwary, Bhavya Patel, Navin Shankar S, Pradeep T J, Balaji A Priyapravas Tarafdar, Mohammed Rafee Information Retrieval Artificial Intelligence Computation and Language I.2.7 Enterprises grapple with the significant challenge of managing proprietary unstructured data, hindering efficient information retrieval. This has led to the emergence of AI-driven information retrieval solutions, designed to adeptly extract relevant insights to address employee inquiries. These solutions often leverage pre-trained embedding models and generative models as foundational components. While pre-trained embeddings may exhibit proximity or disparity based on their original training objectives, they might not fully align with the unique characteristics of enterprise-specific data, leading to suboptimal alignment with the retrieval goals of enterprise environments. In this paper, we propose a comprehensive methodology for contextualizing pre-trained embedding models to enterprise environments, covering the entire process from data preparation to model fine-tuning and evaluation. By adapting the embeddings to better suit the retrieval tasks prevalent in enterprises, we aim to enhance the performance of information retrieval solutions. We discuss the process of fine-tuning, its effect on retrieval accuracy, and the potential benefits for enterprise information management. Our findings demonstrate the efficacy of fine-tuned embedding models in improving the precision and relevance of search results in enterprise settings. |
| title | EnterpriseEM: Fine-tuned Embeddings for Enterprise Semantic Search |
| topic | Information Retrieval Artificial Intelligence Computation and Language I.2.7 |
| url | https://arxiv.org/abs/2406.00010 |