SHE: Stepwise Hybrid Examination Reinforcement Learning Framework for E-commerce Search Relevance
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908958753554432 |
|---|---|
| author | Jiao, Pengkun Jin, Yiming Yang, Jianhui Dong, Chenhe Huang, Zerui Yao, Shaowei Zhou, Xiaojiang Ou, Dan Tang, Haihong |
| author_facet | Jiao, Pengkun Jin, Yiming Yang, Jianhui Dong, Chenhe Huang, Zerui Yao, Shaowei Zhou, Xiaojiang Ou, Dan Tang, Haihong |
| contents | Query-product relevance prediction is vital for AI-driven e-commerce, yet current LLM-based approaches face a dilemma: SFT and DPO struggle with long-tail generalization due to coarse supervision, while traditional RLVR suffers from sparse feedback that fails to correct intermediate reasoning errors. We propose Stepwise Hybrid Examination (SHE), an RL framework that ensures logical consistency through Stepwise Reward Policy Optimization (SRPO). SRPO utilizes a hybrid reward mechanism-combining generative reward models with human-annotated verifiers-to provide fine-grained, step-level signals. To further enhance stability, SHE incorporates diversified data filtering to maintain policy entropy and a multi-stage curriculum learning protocol for progressive skill acquisition. Extensive experiments on real-world search benchmarks show that SHE improves both reasoning quality and relevance-prediction accuracy in large-scale e-commerce settings, outperforming SFT, DPO, GRPO, and other baselines, while also enhancing interpretability and robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_07972 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SHE: Stepwise Hybrid Examination Reinforcement Learning Framework for E-commerce Search Relevance Jiao, Pengkun Jin, Yiming Yang, Jianhui Dong, Chenhe Huang, Zerui Yao, Shaowei Zhou, Xiaojiang Ou, Dan Tang, Haihong Artificial Intelligence Query-product relevance prediction is vital for AI-driven e-commerce, yet current LLM-based approaches face a dilemma: SFT and DPO struggle with long-tail generalization due to coarse supervision, while traditional RLVR suffers from sparse feedback that fails to correct intermediate reasoning errors. We propose Stepwise Hybrid Examination (SHE), an RL framework that ensures logical consistency through Stepwise Reward Policy Optimization (SRPO). SRPO utilizes a hybrid reward mechanism-combining generative reward models with human-annotated verifiers-to provide fine-grained, step-level signals. To further enhance stability, SHE incorporates diversified data filtering to maintain policy entropy and a multi-stage curriculum learning protocol for progressive skill acquisition. Extensive experiments on real-world search benchmarks show that SHE improves both reasoning quality and relevance-prediction accuracy in large-scale e-commerce settings, outperforming SFT, DPO, GRPO, and other baselines, while also enhancing interpretability and robustness. |
| title | SHE: Stepwise Hybrid Examination Reinforcement Learning Framework for E-commerce Search Relevance |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.07972 |