From BERT to LLMs: Comparing and Understanding Chinese Classifier Prediction in Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909881474220032 |
|---|---|
| author | Zhang, Ziqi Ma, Jianfei Chersoni, Emmanuele You, Jieshun Feng, Zhaoxin |
| author_facet | Zhang, Ziqi Ma, Jianfei Chersoni, Emmanuele You, Jieshun Feng, Zhaoxin |
| contents | Classifiers are an important and defining feature of the Chinese language, and their correct prediction is key to numerous educational applications. Yet, whether the most popular Large Language Models (LLMs) possess proper knowledge the Chinese classifiers is an issue that has largely remain unexplored in the Natural Language Processing (NLP) literature.
To address such a question, we employ various masking strategies to evaluate the LLMs' intrinsic ability, the contribution of different sentence elements, and the working of the attention mechanisms during prediction. Besides, we explore fine-tuning for LLMs to enhance the classifier performance.
Our findings reveal that LLMs perform worse than BERT, even with fine-tuning. The prediction, as expected, greatly benefits from the information about the following noun, which also explains the advantage of models with a bidirectional attention mechanism such as BERT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_18253 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From BERT to LLMs: Comparing and Understanding Chinese Classifier Prediction in Language Models Zhang, Ziqi Ma, Jianfei Chersoni, Emmanuele You, Jieshun Feng, Zhaoxin Computation and Language Classifiers are an important and defining feature of the Chinese language, and their correct prediction is key to numerous educational applications. Yet, whether the most popular Large Language Models (LLMs) possess proper knowledge the Chinese classifiers is an issue that has largely remain unexplored in the Natural Language Processing (NLP) literature. To address such a question, we employ various masking strategies to evaluate the LLMs' intrinsic ability, the contribution of different sentence elements, and the working of the attention mechanisms during prediction. Besides, we explore fine-tuning for LLMs to enhance the classifier performance. Our findings reveal that LLMs perform worse than BERT, even with fine-tuning. The prediction, as expected, greatly benefits from the information about the following noun, which also explains the advantage of models with a bidirectional attention mechanism such as BERT. |
| title | From BERT to LLMs: Comparing and Understanding Chinese Classifier Prediction in Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.18253 |