VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908901490819072 |
|---|---|
| author | Liu, Chonghan Du, Yimin An, Qi He, Xin Zhai, Cunqi Tan, Fei Lin, Weijia Gong, Xiaochun Deng, Yongchao Jia, Shousheng Zhang, Xiangzheng |
| author_facet | Liu, Chonghan Du, Yimin An, Qi He, Xin Zhai, Cunqi Tan, Fei Lin, Weijia Gong, Xiaochun Deng, Yongchao Jia, Shousheng Zhang, Xiangzheng |
| contents | Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances. In this paper, we propose Variable Entropy Policy Optimization (VEPO), which leverages Reinforcement Learning with Verifiable Rewards to incorporate deterministic structural constraints into the policy alignment process. This framework ensures prescribed sequence length, robust format consistency, and rigorous linguistic well formedness, all enforced during training. Central to our approach is a variable entropy mechanism that enables the model to dynamically calibrate the equilibrium between literal fidelity and semantic naturalness by modulating the exploration exploitation manifold. By integrating entropy tempered advantage estimation with asymmetric clipping, VEPO sustains robust exploration while mitigating policy collapse. Empirical evaluations across 90 FLORES-200, COMET-22, chrF directions demonstrate that VEPO yields substantial improvements in both tokenization efficiency and translation quality, bridging the performance gap for underrepresented languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_19152 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models Liu, Chonghan Du, Yimin An, Qi He, Xin Zhai, Cunqi Tan, Fei Lin, Weijia Gong, Xiaochun Deng, Yongchao Jia, Shousheng Zhang, Xiangzheng Computation and Language Artificial Intelligence Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances. In this paper, we propose Variable Entropy Policy Optimization (VEPO), which leverages Reinforcement Learning with Verifiable Rewards to incorporate deterministic structural constraints into the policy alignment process. This framework ensures prescribed sequence length, robust format consistency, and rigorous linguistic well formedness, all enforced during training. Central to our approach is a variable entropy mechanism that enables the model to dynamically calibrate the equilibrium between literal fidelity and semantic naturalness by modulating the exploration exploitation manifold. By integrating entropy tempered advantage estimation with asymmetric clipping, VEPO sustains robust exploration while mitigating policy collapse. Empirical evaluations across 90 FLORES-200, COMET-22, chrF directions demonstrate that VEPO yields substantial improvements in both tokenization efficiency and translation quality, bridging the performance gap for underrepresented languages. |
| title | VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2603.19152 |