LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914285535363072 |
|---|---|
| author | Xiong, Lee Chen, Zhirong Mayuranath, Rahul Qiu, Shangran Ozdemir, Arda Li, Lu Hu, Yang Li, Dave Ren, Jingtao Cheng, Howard Herrera, Fabian Souto Agiza, Ahmed Epshtein, Baruch Aggarwal, Anuj Ulziisaikhan, Julia Wang, Chao Ramasamy, Dinesh Doshi, Parshva Reddy, Sri Overwijk, Arnold |
| author_facet | Xiong, Lee Chen, Zhirong Mayuranath, Rahul Qiu, Shangran Ozdemir, Arda Li, Lu Hu, Yang Li, Dave Ren, Jingtao Cheng, Howard Herrera, Fabian Souto Agiza, Ahmed Epshtein, Baruch Aggarwal, Anuj Ulziisaikhan, Julia Wang, Chao Ramasamy, Dinesh Doshi, Parshva Reddy, Sri Overwijk, Arnold |
| contents | We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_20083 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation Xiong, Lee Chen, Zhirong Mayuranath, Rahul Qiu, Shangran Ozdemir, Arda Li, Lu Hu, Yang Li, Dave Ren, Jingtao Cheng, Howard Herrera, Fabian Souto Agiza, Ahmed Epshtein, Baruch Aggarwal, Anuj Ulziisaikhan, Julia Wang, Chao Ramasamy, Dinesh Doshi, Parshva Reddy, Sri Overwijk, Arnold Information Retrieval Artificial Intelligence Machine Learning We present LLaTTE (LLM-Style Latent Transformers for Temporal Events), a scalable transformer architecture for production ads recommendation. Through systematic experiments, we demonstrate that sequence modeling in recommendation systems follows predictable power-law scaling similar to LLMs. Crucially, we find that semantic features bend the scaling curve: they are a prerequisite for scaling, enabling the model to effectively utilize the capacity of deeper and longer architectures. To realize the benefits of continued scaling under strict latency constraints, we introduce a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model. We demonstrate that upstream improvements transfer predictably to downstream ranking tasks. Deployed as the largest user model at Meta, this multi-stage framework drives a 4.3\% conversion uplift on Facebook Feed and Reels with minimal serving overhead, establishing a practical blueprint for harnessing scaling laws in industrial recommender systems. |
| title | LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation |
| topic | Information Retrieval Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2601.20083 |