OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912867970711552 |
|---|---|
| author | Zhang, Zhaoqi Pei, Haolei Guo, Jun Wang, Tianyu Feng, Yufei Sun, Hui Liu, Shaowei Sun, Aixin |
| author_facet | Zhang, Zhaoqi Pei, Haolei Guo, Jun Wang, Tianyu Feng, Yufei Sun, Hui Liu, Shaowei Sun, Aixin |
| contents | In recommendation systems, scaling up feature-interaction modules (e.g., Wukong, RankMixer) or user-behavior sequence modules (e.g., LONGER) has achieved notable success. However, these efforts typically proceed on separate tracks, which not only hinders bidirectional information exchange but also prevents unified optimization and scaling. In this paper, we propose OneTrans, a unified Transformer backbone that simultaneously performs user-behavior sequence modeling and feature interaction. OneTrans employs a unified tokenizer to convert both sequential and non-sequential attributes into a single token sequence. The stacked OneTrans blocks share parameters across similar sequential tokens while assigning token-specific parameters to non-sequential tokens. Through causal attention and cross-request KV caching, OneTrans enables precomputation and caching of intermediate representations, significantly reducing computational costs during both training and inference. Experimental results on industrial-scale datasets demonstrate that OneTrans scales efficiently with increasing parameters, consistently outperforms strong baselines, and yields a 5.68% lift in per-user GMV in online A/B tests. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_26104 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender Zhang, Zhaoqi Pei, Haolei Guo, Jun Wang, Tianyu Feng, Yufei Sun, Hui Liu, Shaowei Sun, Aixin Information Retrieval In recommendation systems, scaling up feature-interaction modules (e.g., Wukong, RankMixer) or user-behavior sequence modules (e.g., LONGER) has achieved notable success. However, these efforts typically proceed on separate tracks, which not only hinders bidirectional information exchange but also prevents unified optimization and scaling. In this paper, we propose OneTrans, a unified Transformer backbone that simultaneously performs user-behavior sequence modeling and feature interaction. OneTrans employs a unified tokenizer to convert both sequential and non-sequential attributes into a single token sequence. The stacked OneTrans blocks share parameters across similar sequential tokens while assigning token-specific parameters to non-sequential tokens. Through causal attention and cross-request KV caching, OneTrans enables precomputation and caching of intermediate representations, significantly reducing computational costs during both training and inference. Experimental results on industrial-scale datasets demonstrate that OneTrans scales efficiently with increasing parameters, consistently outperforms strong baselines, and yields a 5.68% lift in per-user GMV in online A/B tests. |
| title | OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommender |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2510.26104 |