xGR: Efficient Generative Recommendation Serving at Scale
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909970561236992 |
|---|---|
| author | Sun, Qingxiao Liu, Tongxuan Zhang, Shen Wu, Siyu Yang, Peijun Liang, Haotian Li, Menxin Ma, Xiaolong Liang, Zhiwei Ren, Ziyi Zhang, Minchao Liu, Xinyu Zhang, Ke Qian, Depei Yang, Hailong |
| author_facet | Sun, Qingxiao Liu, Tongxuan Zhang, Shen Wu, Siyu Yang, Peijun Liang, Haotian Li, Menxin Ma, Xiaolong Liang, Zhiwei Ren, Ziyi Zhang, Minchao Liu, Xinyu Zhang, Ke Qian, Depei Yang, Hailong |
| contents | Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of long user-item sequences. Despite employing attention-based architectures, GR's workload differs markedly from that of LLM serving. GR typically processes long prompt while producing short, fixed-length outputs, yet the computational cost of each decode phase is especially high due to the large beam width. In addition, since the beam search involves a vast item space, the sorting overhead becomes particularly time-consuming. We propose xGR, a GR-oriented serving system that meets strict low-latency requirements under highconcurrency scenarios. First, xGR unifies the processing of prefill and decode phases through staged computation and separated KV cache. Second, xGR enables early sorting termination and mask-based item filtering with data structure reuse. Third, xGR reconstructs the overall pipeline to exploit multilevel overlap and multi-stream parallelism. Our experiments with real-world recommendation service datasets demonstrate that xGR achieves at least 3.49x throughput compared to the state-of-the-art baseline under strict latency constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_11529 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | xGR: Efficient Generative Recommendation Serving at Scale Sun, Qingxiao Liu, Tongxuan Zhang, Shen Wu, Siyu Yang, Peijun Liang, Haotian Li, Menxin Ma, Xiaolong Liang, Zhiwei Ren, Ziyi Zhang, Minchao Liu, Xinyu Zhang, Ke Qian, Depei Yang, Hailong Machine Learning Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of long user-item sequences. Despite employing attention-based architectures, GR's workload differs markedly from that of LLM serving. GR typically processes long prompt while producing short, fixed-length outputs, yet the computational cost of each decode phase is especially high due to the large beam width. In addition, since the beam search involves a vast item space, the sorting overhead becomes particularly time-consuming. We propose xGR, a GR-oriented serving system that meets strict low-latency requirements under highconcurrency scenarios. First, xGR unifies the processing of prefill and decode phases through staged computation and separated KV cache. Second, xGR enables early sorting termination and mask-based item filtering with data structure reuse. Third, xGR reconstructs the overall pipeline to exploit multilevel overlap and multi-stream parallelism. Our experiments with real-world recommendation service datasets demonstrate that xGR achieves at least 3.49x throughput compared to the state-of-the-art baseline under strict latency constraints. |
| title | xGR: Efficient Generative Recommendation Serving at Scale |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.11529 |