Linear and Neural Dueling Bandits with Delayed Feedback
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866918523568128000 |
|---|---|
| author | Wang, Xiangyi Lu, Pingchen Mao, Jie Kong, Mingze Hong, Zhi Wang, Zhiyong Dai, Zhongxiang |
| author_facet | Wang, Xiangyi Lu, Pingchen Mao, Jie Kong, Mingze Hong, Zhi Wang, Zhiyong Dai, Zhongxiang |
| contents | Contextual dueling bandits form a cornerstone of preference-based decision-making, with critical applications in
recommender systems and large language model alignment. However, standard algorithms rely on the idealized assumption
of immediate feedback, a condition frequently violated in real-world scenarios such as prompt optimization. This
setting introduces a unique theoretical challenge: unlike linear bandits, dueling bandit estimators lack closed-form
solutions, rendering naive adaptations of standard weighting techniques biased. To address this, we formalize the
problem of Contextual Dueling Bandits with Stochastic Delayed Feedback and propose two novel algorithms: Linear
(LDB-DF) and Neural (NDB-DF) Dueling Bandits with Delayed Feedback. Central to our approach is a novel estimator that
integrates an Inverse Probability Weighting (IPW) mechanism directly into the loss function, ensuring unbiased
correction for delayed or missing feedback. We provide comprehensive theoretical analysis, establishing an
O(d*sqrt(T)) regret bound for the linear setting and sub-linear guarantees for the neural setting. Extensive
experiments on both simulated and real-world datasets demonstrate the effectiveness of our propose. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_26554 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Linear and Neural Dueling Bandits with Delayed Feedback Wang, Xiangyi Lu, Pingchen Mao, Jie Kong, Mingze Hong, Zhi Wang, Zhiyong Dai, Zhongxiang Machine Learning Artificial Intelligence Contextual dueling bandits form a cornerstone of preference-based decision-making, with critical applications in recommender systems and large language model alignment. However, standard algorithms rely on the idealized assumption of immediate feedback, a condition frequently violated in real-world scenarios such as prompt optimization. This setting introduces a unique theoretical challenge: unlike linear bandits, dueling bandit estimators lack closed-form solutions, rendering naive adaptations of standard weighting techniques biased. To address this, we formalize the problem of Contextual Dueling Bandits with Stochastic Delayed Feedback and propose two novel algorithms: Linear (LDB-DF) and Neural (NDB-DF) Dueling Bandits with Delayed Feedback. Central to our approach is a novel estimator that integrates an Inverse Probability Weighting (IPW) mechanism directly into the loss function, ensuring unbiased correction for delayed or missing feedback. We provide comprehensive theoretical analysis, establishing an O(d*sqrt(T)) regret bound for the linear setting and sub-linear guarantees for the neural setting. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness of our propose. |
| title | Linear and Neural Dueling Bandits with Delayed Feedback |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.26554 |