Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Liu, Zikun, Luo, Liang, Li, Qianru, Zhang, Zhengyu, Ling, Wei, Shen, Jingyi, Chen, Zeliang, Huang, Yaning, Huang, Jingxian, Aboelela, Abdallah, Sun, Chonglin, Gu, Feifan, Wu, Fenggang, Qu, Hang, Li, Huayu, Pan, Jill, Pei, Kaidi, Chen, Laming, Jin, Longhao, Huang, Qin, Tang, Tongyi, Puvvada, Varna, Chen, Wenlin, Wei, Xiaohan, Cao, Xu, Yao, Yantao, Jin, Yuan, Pu, Yunchen, Chen, Yuxin, Shen, Zijian, Zhang, Zhengkai, Liang, Dong, Wen, Ellie
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	https://arxiv.org/abs/2604.12110
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

Recent advances in recommendation scaling laws have led to foundation models of unprecedented complexity. While these models offer superior performance, their computational demands make real-time serving impractical, often forcing practitioners to rely on knowledge distillation-compromising serving quality for efficiency. To address this challenge, we present SOLARIS (Speculative Offloading of Latent-bAsed Representation for Inference Scaling), a novel framework inspired by speculative decoding. SOLARIS proactively precomputes user-item interaction embeddings by predicting which user-item pairs are likely to appear in future requests, and asynchronously generating their foundation model representations ahead of time. This approach decouples the costly foundation model inference from the latency-critical serving path, enabling real-time knowledge transfer from models previously considered too expensive for online use. Deployed across Meta's advertising system serving billions of daily requests, SOLARIS achieves 0.67% revenue-driving top-line metrics gain, demonstrating its effectiveness at scale.

Similar Items