Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.18793 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911417378013184 |
|---|---|
| author | Wang, Yejing Zhou, Shengyu Lu, Jinyu Liu, Ziwei Liu, Langming Wang, Maolin Zhang, Wenlin Li, Feng Su, Wenbo Wang, Pengjie Xu, Jian Zhao, Xiangyu |
| author_facet | Wang, Yejing Zhou, Shengyu Lu, Jinyu Liu, Ziwei Liu, Langming Wang, Maolin Zhang, Wenlin Li, Feng Su, Wenbo Wang, Pengjie Xu, Jian Zhao, Xiangyu |
| contents | Generative Recommendation (GR), powered by Large Language Models (LLMs), represents a promising new paradigm for industrial recommender systems. However, their practical application is severely hindered by high inference latency, which makes them infeasible for high-throughput, real-time services and limits their overall business impact. While Speculative Decoding (SD) has been proposed to accelerate the autoregressive generation process, existing implementations introduce new bottlenecks: they typically require separate draft models and model-based verifiers, requiring additional training and increasing the latency overhead. In this paper, we address these challenges with NEZHA, a novel architecture that achieves hyperspeed decoding for GR systems without sacrificing recommendation quality. Specifically, NEZHA integrates a nimble autoregressive draft head directly into the primary model, enabling efficient self-drafting. This design, combined with a specialized input prompt structure, preserves the integrity of sequence-to-sequence generation. Furthermore, to tackle the critical problem of hallucination, a major source of performance degradation, we introduce an efficient, model-free verifier based on a hash set. We demonstrate the effectiveness of NEZHA through extensive experiments on public datasets and have successfully deployed the system on Taobao since October 2025, driving the billion-level advertising revenue and serving hundreds of millions of daily active users. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_18793 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations Wang, Yejing Zhou, Shengyu Lu, Jinyu Liu, Ziwei Liu, Langming Wang, Maolin Zhang, Wenlin Li, Feng Su, Wenbo Wang, Pengjie Xu, Jian Zhao, Xiangyu Artificial Intelligence Machine Learning Generative Recommendation (GR), powered by Large Language Models (LLMs), represents a promising new paradigm for industrial recommender systems. However, their practical application is severely hindered by high inference latency, which makes them infeasible for high-throughput, real-time services and limits their overall business impact. While Speculative Decoding (SD) has been proposed to accelerate the autoregressive generation process, existing implementations introduce new bottlenecks: they typically require separate draft models and model-based verifiers, requiring additional training and increasing the latency overhead. In this paper, we address these challenges with NEZHA, a novel architecture that achieves hyperspeed decoding for GR systems without sacrificing recommendation quality. Specifically, NEZHA integrates a nimble autoregressive draft head directly into the primary model, enabling efficient self-drafting. This design, combined with a specialized input prompt structure, preserves the integrity of sequence-to-sequence generation. Furthermore, to tackle the critical problem of hallucination, a major source of performance degradation, we introduce an efficient, model-free verifier based on a hash set. We demonstrate the effectiveness of NEZHA through extensive experiments on public datasets and have successfully deployed the system on Taobao since October 2025, driving the billion-level advertising revenue and serving hundreds of millions of daily active users. |
| title | NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2511.18793 |