How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909495195598848 |
|---|---|
| author | Jiang, Jiyue Chen, Pengan Chen, Liheng Wang, Sheng Bao, Qinghang Kong, Lingpeng Li, Yu Wu, Chuan |
| author_facet | Jiang, Jiyue Chen, Pengan Chen, Liheng Wang, Sheng Bao, Qinghang Kong, Lingpeng Li, Yu Wu, Chuan |
| contents | The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages. However, underrepresented languages like Cantonese, spoken by over 85 million people, face significant development gaps, which is particularly concerning given the economic significance of the Guangdong-Hong Kong-Macau Greater Bay Area, and in substantial Cantonese-speaking populations in places like Singapore and North America. Despite its wide use, Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. To bridge these gaps, we outline current Cantonese NLP methods and introduce new benchmarks designed to evaluate LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonese, which aim to advance open-source Cantonese LLM technology. We also propose future research directions and recommended models to enhance Cantonese LLM development. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_16756 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models Jiang, Jiyue Chen, Pengan Chen, Liheng Wang, Sheng Bao, Qinghang Kong, Lingpeng Li, Yu Wu, Chuan Computation and Language The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages. However, underrepresented languages like Cantonese, spoken by over 85 million people, face significant development gaps, which is particularly concerning given the economic significance of the Guangdong-Hong Kong-Macau Greater Bay Area, and in substantial Cantonese-speaking populations in places like Singapore and North America. Despite its wide use, Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. To bridge these gaps, we outline current Cantonese NLP methods and introduce new benchmarks designed to evaluate LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonese, which aim to advance open-source Cantonese LLM technology. We also propose future research directions and recommended models to enhance Cantonese LLM development. |
| title | How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.16756 |