Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917223929479168 |
|---|---|
| author | Sutawika, Lintang Swamy, Gokul Wu, Zhiwei Steven Neubig, Graham |
| author_facet | Sutawika, Lintang Swamy, Gokul Wu, Zhiwei Steven Neubig, Graham |
| contents | When asked a question in a language less seen in its training data, current reasoning large language models (RLMs) often exhibit dramatically lower performance than when asked the same question in English. In response, we introduce \texttt{SP3F} (Self-Play with Privileged Pairwise Feedback), a two-stage framework for enhancing multilingual reasoning without \textit{any} data in the target language(s). First, we supervise fine-tune (SFT) on translated versions of English question-answer pairs to raise base model correctness. Second, we perform RL with feedback from a pairwise judge in a self-play fashion, with the judge receiving the English reference response as \textit{privileged information}. Thus, even when none of the model's responses are completely correct, the privileged pairwise judge can still tell which response is better. End-to-end, \texttt{SP3F} greatly improves base model performance, even outperforming fully post-trained models on multiple math and non-math tasks with less than
of the training data across the single-language, multilingual, and generalization to unseen language settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_18722 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning Sutawika, Lintang Swamy, Gokul Wu, Zhiwei Steven Neubig, Graham Computation and Language Machine Learning When asked a question in a language less seen in its training data, current reasoning large language models (RLMs) often exhibit dramatically lower performance than when asked the same question in English. In response, we introduce \texttt{SP3F} (Self-Play with Privileged Pairwise Feedback), a two-stage framework for enhancing multilingual reasoning without \textit{any} data in the target language(s). First, we supervise fine-tune (SFT) on translated versions of English question-answer pairs to raise base model correctness. Second, we perform RL with feedback from a pairwise judge in a self-play fashion, with the judge receiving the English reference response as \textit{privileged information}. Thus, even when none of the model's responses are completely correct, the privileged pairwise judge can still tell which response is better. End-to-end, \texttt{SP3F} greatly improves base model performance, even outperforming fully post-trained models on multiple math and non-math tasks with less than of the training data across the single-language, multilingual, and generalization to unseen language settings. |
| title | Gained in Translation: Privileged Pairwise Judges Enhance Multilingual Reasoning |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2601.18722 |