Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918065721049088 |
|---|---|
| author | Ghosal, Soumya Suvra Singh, Vaibhav Ghosh, Akash Pal, Soumyabrata Baidya, Subhadip Saha, Sriparna Manocha, Dinesh |
| author_facet | Ghosal, Soumya Suvra Singh, Vaibhav Ghosh, Akash Pal, Soumyabrata Baidya, Subhadip Saha, Sriparna Manocha, Dinesh |
| contents | Reward models are essential for aligning large language models (LLMs) with human preferences. However, most open-source multilingual reward models are primarily trained on preference datasets in high-resource languages, resulting in unreliable reward signals for low-resource Indic languages. Collecting large-scale, high-quality preference data for these languages is prohibitively expensive, making preference-based training approaches impractical. To address this challenge, we propose RELIC, a novel in-context learning framework for reward modeling in low-resource Indic languages. RELIC trains a retriever with a pairwise ranking objective to select in-context examples from auxiliary high-resource languages that most effectively highlight the distinction between preferred and less-preferred responses. Extensive experiments on three preference datasets- PKU-SafeRLHF, WebGPT, and HH-RLHF-using state-of-the-art open-source reward models demonstrate that RELIC significantly improves reward model accuracy for low-resource Indic languages, consistently outperforming existing example selection methods. For example, on Bodo-a low-resource Indic language-using a LLaMA-3.2-3B reward model, RELIC achieves a 12.81% and 10.13% improvement in accuracy over zero-shot prompting and state-of-the-art example selection method, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_16502 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples Ghosal, Soumya Suvra Singh, Vaibhav Ghosh, Akash Pal, Soumyabrata Baidya, Subhadip Saha, Sriparna Manocha, Dinesh Computation and Language Artificial Intelligence Reward models are essential for aligning large language models (LLMs) with human preferences. However, most open-source multilingual reward models are primarily trained on preference datasets in high-resource languages, resulting in unreliable reward signals for low-resource Indic languages. Collecting large-scale, high-quality preference data for these languages is prohibitively expensive, making preference-based training approaches impractical. To address this challenge, we propose RELIC, a novel in-context learning framework for reward modeling in low-resource Indic languages. RELIC trains a retriever with a pairwise ranking objective to select in-context examples from auxiliary high-resource languages that most effectively highlight the distinction between preferred and less-preferred responses. Extensive experiments on three preference datasets- PKU-SafeRLHF, WebGPT, and HH-RLHF-using state-of-the-art open-source reward models demonstrate that RELIC significantly improves reward model accuracy for low-resource Indic languages, consistently outperforming existing example selection methods. For example, on Bodo-a low-resource Indic language-using a LLaMA-3.2-3B reward model, RELIC achieves a 12.81% and 10.13% improvement in accuracy over zero-shot prompting and state-of-the-art example selection method, respectively. |
| title | Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.16502 |