Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2504.02268 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910902548168704 |
|---|---|
| author | Gill, Waris Cechmanek, Justin Hutcherson, Tyler Rajamohan, Srijith Agarwal, Jen Gulzar, Muhammad Ali Singh, Manvinder Dion, Benoit |
| author_facet | Gill, Waris Cechmanek, Justin Hutcherson, Tyler Rajamohan, Srijith Agarwal, Jen Gulzar, Muhammad Ali Singh, Manvinder Dion, Benoit |
| contents | This report investigates enhancing semantic caching effectiveness by employing specialized, fine-tuned embedding models. Semantic caching relies on embedding similarity rather than exact key matching, presenting unique challenges in balancing precision, query latency, and computational efficiency. We propose leveraging smaller, domain-specific embedding models, fine-tuned with targeted real-world and synthetically generated datasets. Our empirical evaluations demonstrate that compact embedding models fine-tuned for just one epoch on specialized datasets significantly surpass both state-of-the-art open-source and proprietary alternatives in precision and recall. Moreover, we introduce a novel synthetic data generation pipeline for the semantic cache that mitigates the challenge of limited domain-specific annotated data, further boosting embedding performance. Our approach effectively balances computational overhead and accuracy, establishing a viable and efficient strategy for practical semantic caching implementations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_02268 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data Gill, Waris Cechmanek, Justin Hutcherson, Tyler Rajamohan, Srijith Agarwal, Jen Gulzar, Muhammad Ali Singh, Manvinder Dion, Benoit Machine Learning Computation and Language This report investigates enhancing semantic caching effectiveness by employing specialized, fine-tuned embedding models. Semantic caching relies on embedding similarity rather than exact key matching, presenting unique challenges in balancing precision, query latency, and computational efficiency. We propose leveraging smaller, domain-specific embedding models, fine-tuned with targeted real-world and synthetically generated datasets. Our empirical evaluations demonstrate that compact embedding models fine-tuned for just one epoch on specialized datasets significantly surpass both state-of-the-art open-source and proprietary alternatives in precision and recall. Moreover, we introduce a novel synthetic data generation pipeline for the semantic cache that mitigates the challenge of limited domain-specific annotated data, further boosting embedding performance. Our approach effectively balances computational overhead and accuracy, establishing a viable and efficient strategy for practical semantic caching implementations. |
| title | Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2504.02268 |