Post-Training Language Models for Crosslingual Consistency
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917543659175936 |
|---|---|
| author | Liu, Tianyu Qi, Jirui Sachan, Mrinmaya Cotterell, Ryan Fernández, Raquel Bisazza, Arianna |
| author_facet | Liu, Tianyu Qi, Jirui Sachan, Mrinmaya Cotterell, Ryan Fernández, Raquel Bisazza, Arianna |
| contents | Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model's response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback-Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_04678 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Post-Training Language Models for Crosslingual Consistency Liu, Tianyu Qi, Jirui Sachan, Mrinmaya Cotterell, Ryan Fernández, Raquel Bisazza, Arianna Computation and Language Artificial Intelligence Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model's response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback-Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages. |
| title | Post-Training Language Models for Crosslingual Consistency |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2603.04678 |