Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908439875158016 |
|---|---|
| author | Attimonelli, Matteo De Bellis, Alessandro Pomo, Claudio Jannach, Dietmar Di Sciascio, Eugenio Di Noia, Tommaso |
| author_facet | Attimonelli, Matteo De Bellis, Alessandro Pomo, Claudio Jannach, Dietmar Di Sciascio, Eugenio Di Noia, Tommaso |
| contents | Pre-trained language models (PLMs) are widely used to derive semantic representations from item metadata in recommendation and search. In sequential recommendation, PLMs enhance ID-based embeddings through textual metadata, while in product search, they align item characteristics with user intent. Recent studies suggest task and domain-specific fine-tuning are needed to improve representational power. This paper challenges this assumption, showing that Generalist Text Embedding Models (GTEs), pre-trained on large-scale corpora, can guarantee strong zero-shot performance without specialized adaptation. Our experiments demonstrate that GTEs outperform traditional and fine-tuned models in both sequential recommendation and product search. We attribute this to a superior representational power, as they distribute features more evenly across the embedding space. Finally, we show that compressing embedding dimensions by focusing on the most informative directions (e.g., via PCA) effectively reduces noise and improves the performance of specialized models. To ensure reproducibility, we provide our repository at https://split.to/gte4ps. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_05006 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search Attimonelli, Matteo De Bellis, Alessandro Pomo, Claudio Jannach, Dietmar Di Sciascio, Eugenio Di Noia, Tommaso Information Retrieval Computation and Language Pre-trained language models (PLMs) are widely used to derive semantic representations from item metadata in recommendation and search. In sequential recommendation, PLMs enhance ID-based embeddings through textual metadata, while in product search, they align item characteristics with user intent. Recent studies suggest task and domain-specific fine-tuning are needed to improve representational power. This paper challenges this assumption, showing that Generalist Text Embedding Models (GTEs), pre-trained on large-scale corpora, can guarantee strong zero-shot performance without specialized adaptation. Our experiments demonstrate that GTEs outperform traditional and fine-tuned models in both sequential recommendation and product search. We attribute this to a superior representational power, as they distribute features more evenly across the embedding space. Finally, we show that compressing embedding dimensions by focusing on the most informative directions (e.g., via PCA) effectively reduces noise and improves the performance of specialized models. To ensure reproducibility, we provide our repository at https://split.to/gte4ps. |
| title | Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2507.05006 |