LinkedIn Post Embeddings: Industrial Scale Embedding Generation and Usage across LinkedIn
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914100166000640 |
|---|---|
| author | Ramanujam, Sudarshan Srinivasa Bindal, Akanksha Jiang, Yu Hazen, Timothy J. Golland, David Zhang, Fengyu Sun, Daqi Li, Wanning Tiwana, Birjodh Singh Dangi, Siddharth Yan, Peng |
| author_facet | Ramanujam, Sudarshan Srinivasa Bindal, Akanksha Jiang, Yu Hazen, Timothy J. Golland, David Zhang, Fengyu Sun, Daqi Li, Wanning Tiwana, Birjodh Singh Dangi, Siddharth Yan, Peng |
| contents | A post embedding (representation of text in embedding space that effectively captures semantic meaning) is a foundational component of LinkedIn that is consumed by product surfaces in retrieval and ranking (e.g., ranking posts in the feed or video tab). This paper presents the post embeddings used at LinkedIn, where a pre-trained transformer-based large language model (LLM) is taken as input and fine-tuned using multi-task learning across a diverse set of semantic labeling tasks. We observe positive transfer, leading to improved performance across all tasks, compared to training them independently. The generated post embeddings outperform baseline models in zero-shot learning, demonstrating its potential for broader applicability. Furthermore, the generated post embeddings' performance surpasses that of OpenAI's ADA-001 and ADA-002 embeddings on LinkedIn specific datasets and tasks. We also describe the offline evaluation methodology and the deployment to our near-line infrastructure, which makes the post embedding available for use within minutes of post creation for any downstream application. We present how the embeddings were applied in the Feed product surface, in both ranking and retrieval stages, and showcase the real world online impact to demonstrate the superior performance of these embeddings. Finally, we also share the results of applying the embeddings to the retrieval system of our video ranking product surface in LinkedIn. These embeddings have been battle-tested in production at LinkedIn for over two years, consistently powering multiple products. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_11344 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LinkedIn Post Embeddings: Industrial Scale Embedding Generation and Usage across LinkedIn Ramanujam, Sudarshan Srinivasa Bindal, Akanksha Jiang, Yu Hazen, Timothy J. Golland, David Zhang, Fengyu Sun, Daqi Li, Wanning Tiwana, Birjodh Singh Dangi, Siddharth Yan, Peng Machine Learning Artificial Intelligence A post embedding (representation of text in embedding space that effectively captures semantic meaning) is a foundational component of LinkedIn that is consumed by product surfaces in retrieval and ranking (e.g., ranking posts in the feed or video tab). This paper presents the post embeddings used at LinkedIn, where a pre-trained transformer-based large language model (LLM) is taken as input and fine-tuned using multi-task learning across a diverse set of semantic labeling tasks. We observe positive transfer, leading to improved performance across all tasks, compared to training them independently. The generated post embeddings outperform baseline models in zero-shot learning, demonstrating its potential for broader applicability. Furthermore, the generated post embeddings' performance surpasses that of OpenAI's ADA-001 and ADA-002 embeddings on LinkedIn specific datasets and tasks. We also describe the offline evaluation methodology and the deployment to our near-line infrastructure, which makes the post embedding available for use within minutes of post creation for any downstream application. We present how the embeddings were applied in the Feed product surface, in both ranking and retrieval stages, and showcase the real world online impact to demonstrate the superior performance of these embeddings. Finally, we also share the results of applying the embeddings to the retrieval system of our video ranking product surface in LinkedIn. These embeddings have been battle-tested in production at LinkedIn for over two years, consistently powering multiple products. |
| title | LinkedIn Post Embeddings: Industrial Scale Embedding Generation and Usage across LinkedIn |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2405.11344 |