Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2411.16387 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910713372475392 |
|---|---|
| author | Lin, Cheng-Wei Hsieh, Wan-Hsuan Guan, Kai-Xin Hsu, Chan-Jan Kuo, Chia-Chen Lai, Chuan-Lin Chung, Chung-Wei Wang, Ming-Jen Shiu, Da-Shan |
| author_facet | Lin, Cheng-Wei Hsieh, Wan-Hsuan Guan, Kai-Xin Hsu, Chan-Jan Kuo, Chia-Chen Lai, Chuan-Lin Chung, Chung-Wei Wang, Ming-Jen Shiu, Da-Shan |
| contents | The quality and size of a pretraining dataset significantly influence the performance of large language models (LLMs). While there have been numerous efforts in the curation of such a dataset for English users, there is a relative lack of similar initiatives for Traditional Chinese. Building upon this foundation of FineWeb, we introduce FineWeb-zhtw, a dataset tailored specifically for Traditional Chinese users. We came up with multiple stages of meticulously designed filters to cater to the linguistic difference between English and Traditional Chinese, to ensure comprehensiveness and quality. We determined effectiveness from querying dataset samples with three main objectives. Our code and datasets are publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_16387 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FineWeb-zhtw: Scalable Curation of Traditional Chinese Text Data from the Web Lin, Cheng-Wei Hsieh, Wan-Hsuan Guan, Kai-Xin Hsu, Chan-Jan Kuo, Chia-Chen Lai, Chuan-Lin Chung, Chung-Wei Wang, Ming-Jen Shiu, Da-Shan Computation and Language Databases The quality and size of a pretraining dataset significantly influence the performance of large language models (LLMs). While there have been numerous efforts in the curation of such a dataset for English users, there is a relative lack of similar initiatives for Traditional Chinese. Building upon this foundation of FineWeb, we introduce FineWeb-zhtw, a dataset tailored specifically for Traditional Chinese users. We came up with multiple stages of meticulously designed filters to cater to the linguistic difference between English and Traditional Chinese, to ensure comprehensiveness and quality. We determined effectiveness from querying dataset samples with three main objectives. Our code and datasets are publicly available. |
| title | FineWeb-zhtw: Scalable Curation of Traditional Chinese Text Data from the Web |
| topic | Computation and Language Databases |
| url | https://arxiv.org/abs/2411.16387 |