Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.18511 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918257490919424 |
|---|---|
| author | Wang, Siyi Wang, Zifan Johanssson, Karl Henrik |
| author_facet | Wang, Siyi Wang, Zifan Johanssson, Karl Henrik |
| contents | In interactive systems, feedback is often provided in the form of preference between queried options rather than precise scores, which motivates optimization methods to learn from such comparisons. In this work, we propose a preference-based optimization algorithm that relies on noisy two-point comparisons. At each iteration, the algorithm employs a uniform-sphere perturbation to generate a perturbed action and queries the resulting loss comparison to estimate a descent direction. We demonstrate that, under standard smoothness and bounded variance assumptions, the algorithm converges to a stationary point when the smoothing and step size parameters are properly chosen. Numerical experiments on an LQG system demonstrate the effectiveness of the preference-based optimization algorithm with comparison feedback. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_18511 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Preference-based optimization from noisy pairwise comparisons Wang, Siyi Wang, Zifan Johanssson, Karl Henrik Optimization and Control In interactive systems, feedback is often provided in the form of preference between queried options rather than precise scores, which motivates optimization methods to learn from such comparisons. In this work, we propose a preference-based optimization algorithm that relies on noisy two-point comparisons. At each iteration, the algorithm employs a uniform-sphere perturbation to generate a perturbed action and queries the resulting loss comparison to estimate a descent direction. We demonstrate that, under standard smoothness and bounded variance assumptions, the algorithm converges to a stationary point when the smoothing and step size parameters are properly chosen. Numerical experiments on an LQG system demonstrate the effectiveness of the preference-based optimization algorithm with comparison feedback. |
| title | Preference-based optimization from noisy pairwise comparisons |
| topic | Optimization and Control |
| url | https://arxiv.org/abs/2512.18511 |