RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913115174600704 |
|---|---|
| author | Chen, Jin Zhang, Shangyu Hu, Bin Zhou, Chao Pan, Junwei Xue, Gengsheng Ning, Wentao Weng, Gengyu Zheng, Wang Liu, Shaohua Xu, Zeen Mai, Chengyuan Quan, Shijie Jiang, Tingyu Wang, Lifeng Huang, Shudong Yin, Chengguo Gu, Haijie Jiang, Jie |
| author_facet | Chen, Jin Zhang, Shangyu Hu, Bin Zhou, Chao Pan, Junwei Xue, Gengsheng Ning, Wentao Weng, Gengyu Zheng, Wang Liu, Shaohua Xu, Zeen Mai, Chengyuan Quan, Shijie Jiang, Tingyu Wang, Lifeng Huang, Shudong Yin, Chengguo Gu, Haijie Jiang, Jie |
| contents | The scaling laws for recommender systems have been increasingly validated, where MetaFormer-based architectures consistently benefit from increased model depth, hidden dimensionality, and user behavior sequence length. However, whether representation capacity scales proportionally with parameter growth remains unexplored. Prior studies on RankMixer reveal that the effective rank of token representations exhibits a damped oscillatory trajectory across layers, failing to increase consistently with depth and even degrading in deeper layers. Motivated by this observation, we propose RankUp, an architecture designed to mitigate representation collapse and enhance expressive capacity through randomized permutation splitting over sparse features, a multi-embedding paradigm, global token integration and crossed pretrained embedding tokens. RankUp has been fully deployed in large-scale production across Weixin Video Accounts, Official Accounts and Moments, yielding GMV improvements of 3.41%, 4.81% and 2.12%, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_17878 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems Chen, Jin Zhang, Shangyu Hu, Bin Zhou, Chao Pan, Junwei Xue, Gengsheng Ning, Wentao Weng, Gengyu Zheng, Wang Liu, Shaohua Xu, Zeen Mai, Chengyuan Quan, Shijie Jiang, Tingyu Wang, Lifeng Huang, Shudong Yin, Chengguo Gu, Haijie Jiang, Jie Information Retrieval The scaling laws for recommender systems have been increasingly validated, where MetaFormer-based architectures consistently benefit from increased model depth, hidden dimensionality, and user behavior sequence length. However, whether representation capacity scales proportionally with parameter growth remains unexplored. Prior studies on RankMixer reveal that the effective rank of token representations exhibits a damped oscillatory trajectory across layers, failing to increase consistently with depth and even degrading in deeper layers. Motivated by this observation, we propose RankUp, an architecture designed to mitigate representation collapse and enhance expressive capacity through randomized permutation splitting over sparse features, a multi-embedding paradigm, global token integration and crossed pretrained embedding tokens. RankUp has been fully deployed in large-scale production across Weixin Video Accounts, Official Accounts and Moments, yielding GMV improvements of 3.41%, 4.81% and 2.12%, respectively. |
| title | RankUp: Towards High-rank Representations for Large Scale Advertising Recommender Systems |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2604.17878 |