Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.29259 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914433694957568 |
|---|---|
| author | Huang, Hejin Zhang, Jusheng Cai, Kaitong Wang, Jian Pan, Rong |
| author_facet | Huang, Hejin Zhang, Jusheng Cai, Kaitong Wang, Jian Pan, Rong |
| contents | Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, existing work rarely examines how Direct Preference Optimization (DPO) behaves under implicit feedback, where unobserved items are not reliable negatives. We conduct systematic experiments on multimodal sequential recommendation to compare common negative-selection strategies and their interaction with DPO training. Our central finding is that a simple modification, replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool, consistently improves ranking performance. We attribute its effectiveness to two factors: (1) reducing erroneous suppressive gradients caused by false negatives, and (2) retaining informative hard signals while smoothing optimization via controlled stochasticity. With an optional sparse Mixture-of-Experts encoder for efficient capacity scaling, RoDPO achieves up to 5.25% NDCG@5 on three Amazon benchmarks, with nearly unchanged inference cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_29259 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE Huang, Hejin Zhang, Jusheng Cai, Kaitong Wang, Jian Pan, Rong Information Retrieval Computation and Language Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, existing work rarely examines how Direct Preference Optimization (DPO) behaves under implicit feedback, where unobserved items are not reliable negatives. We conduct systematic experiments on multimodal sequential recommendation to compare common negative-selection strategies and their interaction with DPO training. Our central finding is that a simple modification, replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool, consistently improves ranking performance. We attribute its effectiveness to two factors: (1) reducing erroneous suppressive gradients caused by false negatives, and (2) retaining informative hard signals while smoothing optimization via controlled stochasticity. With an optional sparse Mixture-of-Experts encoder for efficient capacity scaling, RoDPO achieves up to 5.25% NDCG@5 on three Amazon benchmarks, with nearly unchanged inference cost. |
| title | Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE |
| topic | Information Retrieval Computation and Language |
| url | https://arxiv.org/abs/2603.29259 |