Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929594341261312 |
|---|---|
| author | Luo, Xianzhen Zhu, Qingfu Zhang, Zhiming Qin, Libo Zhang, Xuanyu Yang, Qing Xu, Dongliang Che, Wanxiang |
| author_facet | Luo, Xianzhen Zhu, Qingfu Zhang, Zhiming Qin, Libo Zhang, Xuanyu Yang, Qing Xu, Dongliang Che, Wanxiang |
| contents | Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Currently, PoT primarily uses Python. However, relying solely on a single language may result in suboptimal solutions and overlook the potential benefits of other programming languages. In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models. The effectiveness of each language varies depending on the specific scenarios. Inspired by this, we propose a task and model agnostic approach called MultiPoT, which harnesses strength and diversity from various languages. Experimental results reveal that it significantly outperforms Python Self-Consistency. Furthermore, it achieves comparable or superior performance compared to the best monolingual PoT in almost all tasks across all models. In particular, MultiPoT achieves more than 4.6% improvement on average on ChatGPT (gpt-3.5-turbo-0701). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10691 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts Luo, Xianzhen Zhu, Qingfu Zhang, Zhiming Qin, Libo Zhang, Xuanyu Yang, Qing Xu, Dongliang Che, Wanxiang Computation and Language Program of Thoughts (PoT) is an approach characterized by its executable intermediate steps, which ensure the accuracy of the logical calculations in the reasoning process. Currently, PoT primarily uses Python. However, relying solely on a single language may result in suboptimal solutions and overlook the potential benefits of other programming languages. In this paper, we conduct comprehensive experiments on the programming languages used in PoT and find that no single language consistently delivers optimal performance across all tasks and models. The effectiveness of each language varies depending on the specific scenarios. Inspired by this, we propose a task and model agnostic approach called MultiPoT, which harnesses strength and diversity from various languages. Experimental results reveal that it significantly outperforms Python Self-Consistency. Furthermore, it achieves comparable or superior performance compared to the best monolingual PoT in almost all tasks across all models. In particular, MultiPoT achieves more than 4.6% improvement on average on ChatGPT (gpt-3.5-turbo-0701). |
| title | Python is Not Always the Best Choice: Embracing Multilingual Program of Thoughts |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2402.10691 |