Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2403.18058 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910681042780160 |
|---|---|
| author | Bai, Yuelin Du, Xinrun Liang, Yiming Jin, Yonggang Zhou, Junting Liu, Ziqiang Fang, Feiteng Chang, Mingshan Zheng, Tianyu Zhang, Xincheng Ma, Nuo Wang, Zekun Yuan, Ruibin Wu, Haihong Lin, Hongquan Huang, Wenhao Zhang, Jiajun Lin, Chenghua Fu, Jie Yang, Min Ni, Shiwen Zhang, Ge |
| author_facet | Bai, Yuelin Du, Xinrun Liang, Yiming Jin, Yonggang Zhou, Junting Liu, Ziqiang Fang, Feiteng Chang, Mingshan Zheng, Tianyu Zhang, Xincheng Ma, Nuo Wang, Zekun Yuan, Ruibin Wu, Haihong Lin, Hongquan Huang, Wenhao Zhang, Jiajun Lin, Chenghua Fu, Jie Yang, Min Ni, Shiwen Zhang, Ge |
| contents | Remarkable progress on English instruction tuning has facilitated the efficacy and reliability of large language models (LLMs). However, there remains a noticeable gap in instruction tuning for Chinese, where the complex linguistic features pose significant challenges. Existing datasets, generally distilled from English-centric LLMs, are not well-aligned with Chinese users' interaction patterns. To bridge this gap, we introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world resources and undergoing rigorous human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks. Additionally, our findings offer several insights for designing effective Chinese instruction-tuning datasets and data-mixing strategies. Our dataset are available at https://huggingface.co/datasets/m-a-p/COIG-CQIA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_18058 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning Bai, Yuelin Du, Xinrun Liang, Yiming Jin, Yonggang Zhou, Junting Liu, Ziqiang Fang, Feiteng Chang, Mingshan Zheng, Tianyu Zhang, Xincheng Ma, Nuo Wang, Zekun Yuan, Ruibin Wu, Haihong Lin, Hongquan Huang, Wenhao Zhang, Jiajun Lin, Chenghua Fu, Jie Yang, Min Ni, Shiwen Zhang, Ge Computation and Language Artificial Intelligence Remarkable progress on English instruction tuning has facilitated the efficacy and reliability of large language models (LLMs). However, there remains a noticeable gap in instruction tuning for Chinese, where the complex linguistic features pose significant challenges. Existing datasets, generally distilled from English-centric LLMs, are not well-aligned with Chinese users' interaction patterns. To bridge this gap, we introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world resources and undergoing rigorous human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks. Additionally, our findings offer several insights for designing effective Chinese instruction-tuning datasets and data-mixing strategies. Our dataset are available at https://huggingface.co/datasets/m-a-p/COIG-CQIA. |
| title | COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.18058 |