LangBridge: Multilingual Reasoning Without Multilingual Supervision
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909215083200512 |
|---|---|
| author | Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon |
| author_facet | Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon |
| contents | We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects: (1) one specialized in understanding multiple languages (e.g., mT5 encoder) and (2) one specialized in reasoning (e.g., MetaMath). LangBridge connects the two models by introducing minimal trainable parameters between them. Despite utilizing only English data for training, LangBridge considerably enhances the performance of language models on low-resource languages across mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. Our analysis suggests that the efficacy of LangBridge stems from the language-agnostic characteristics of multilingual representations. We publicly release our code and models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_10695 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LangBridge: Multilingual Reasoning Without Multilingual Supervision Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon Computation and Language We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects: (1) one specialized in understanding multiple languages (e.g., mT5 encoder) and (2) one specialized in reasoning (e.g., MetaMath). LangBridge connects the two models by introducing minimal trainable parameters between them. Despite utilizing only English data for training, LangBridge considerably enhances the performance of language models on low-resource languages across mathematical reasoning, code completion, logical reasoning, and commonsense reasoning. Our analysis suggests that the efficacy of LangBridge stems from the language-agnostic characteristics of multilingual representations. We publicly release our code and models. |
| title | LangBridge: Multilingual Reasoning Without Multilingual Supervision |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.10695 |