Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.12110 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913029013110784 |
|---|---|
| author | Liu, Zikun Luo, Liang Li, Qianru Zhang, Zhengyu Ling, Wei Shen, Jingyi Chen, Zeliang Huang, Yaning Huang, Jingxian Aboelela, Abdallah Sun, Chonglin Gu, Feifan Wu, Fenggang Qu, Hang Li, Huayu Pan, Jill Pei, Kaidi Chen, Laming Jin, Longhao Huang, Qin Tang, Tongyi Puvvada, Varna Chen, Wenlin Wei, Xiaohan Cao, Xu Yao, Yantao Jin, Yuan Pu, Yunchen Chen, Yuxin Shen, Zijian Zhang, Zhengkai Liang, Dong Wen, Ellie |
| author_facet | Liu, Zikun Luo, Liang Li, Qianru Zhang, Zhengyu Ling, Wei Shen, Jingyi Chen, Zeliang Huang, Yaning Huang, Jingxian Aboelela, Abdallah Sun, Chonglin Gu, Feifan Wu, Fenggang Qu, Hang Li, Huayu Pan, Jill Pei, Kaidi Chen, Laming Jin, Longhao Huang, Qin Tang, Tongyi Puvvada, Varna Chen, Wenlin Wei, Xiaohan Cao, Xu Yao, Yantao Jin, Yuan Pu, Yunchen Chen, Yuxin Shen, Zijian Zhang, Zhengkai Liang, Dong Wen, Ellie |
| contents | Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity. While these models offer superior performance, their computational demands make real-time serving impractical, often forcing practitioners to rely on knowledge distillation-compromising serving quality for efficiency. To address this challenge, we present SOLARIS (Speculative Offloading of Latent-bAsed Representation for Inference Scaling), a novel framework inspired by speculative decoding. SOLARIS proactively precomputes user-item interaction embeddings by predicting which user-item pairs are likely to appear in future requests, and asynchronously generating their foundation model representations ahead of time. This approach decouples the costly foundation model inference from the latency-critical serving path, enabling real-time knowledge transfer from models previously considered too expensive for online use. Deployed across Meta's advertising system serving billions of daily requests, SOLARIS achieves 0.67% revenue-driving top-line metrics gain, demonstrating its effectiveness at scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_12110 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling Liu, Zikun Luo, Liang Li, Qianru Zhang, Zhengyu Ling, Wei Shen, Jingyi Chen, Zeliang Huang, Yaning Huang, Jingxian Aboelela, Abdallah Sun, Chonglin Gu, Feifan Wu, Fenggang Qu, Hang Li, Huayu Pan, Jill Pei, Kaidi Chen, Laming Jin, Longhao Huang, Qin Tang, Tongyi Puvvada, Varna Chen, Wenlin Wei, Xiaohan Cao, Xu Yao, Yantao Jin, Yuan Pu, Yunchen Chen, Yuxin Shen, Zijian Zhang, Zhengkai Liang, Dong Wen, Ellie Machine Learning Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity. While these models offer superior performance, their computational demands make real-time serving impractical, often forcing practitioners to rely on knowledge distillation-compromising serving quality for efficiency. To address this challenge, we present SOLARIS (Speculative Offloading of Latent-bAsed Representation for Inference Scaling), a novel framework inspired by speculative decoding. SOLARIS proactively precomputes user-item interaction embeddings by predicting which user-item pairs are likely to appear in future requests, and asynchronously generating their foundation model representations ahead of time. This approach decouples the costly foundation model inference from the latency-critical serving path, enabling real-time knowledge transfer from models previously considered too expensive for online use. Deployed across Meta's advertising system serving billions of daily requests, SOLARIS achieves 0.67% revenue-driving top-line metrics gain, demonstrating its effectiveness at scale. |
| title | SOLARIS: Speculative Offloading of Latent-bAsed Representation for Inference Scaling |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2604.12110 |