An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915349227634688 |
|---|---|
| author | Tang, Xuemei Wang, Jun Su, Qi Huang, Chu-ren Gu, Jinghang |
| author_facet | Tang, Xuemei Wang, Jun Su, Qi Huang, Chu-ren Gu, Jinghang |
| contents | Sequence labeling models often benefit from incorporating external knowledge. However, this practice introduces data heterogeneity and complicates the model with additional modules, leading to increased expenses for training a high-performing model. To address this challenge, we propose a two-stage curriculum learning (TCL) framework specifically designed for sequence labeling tasks. The TCL framework enhances training by gradually introducing data instances from easy to hard, aiming to improve both performance and training speed. Furthermore, we explore different metrics for assessing the difficulty levels of sequence labeling tasks. Through extensive experimentation on six Chinese word segmentation (CWS) and Part-of-speech tagging (POS) datasets, we demonstrate the effectiveness of our model in enhancing the performance of sequence labeling models. Additionally, our analysis indicates that TCL accelerates training and alleviates the slow training problem associated with complex models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13534 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling Tang, Xuemei Wang, Jun Su, Qi Huang, Chu-ren Gu, Jinghang Computation and Language Artificial Intelligence Sequence labeling models often benefit from incorporating external knowledge. However, this practice introduces data heterogeneity and complicates the model with additional modules, leading to increased expenses for training a high-performing model. To address this challenge, we propose a two-stage curriculum learning (TCL) framework specifically designed for sequence labeling tasks. The TCL framework enhances training by gradually introducing data instances from easy to hard, aiming to improve both performance and training speed. Furthermore, we explore different metrics for assessing the difficulty levels of sequence labeling tasks. Through extensive experimentation on six Chinese word segmentation (CWS) and Part-of-speech tagging (POS) datasets, we demonstrate the effectiveness of our model in enhancing the performance of sequence labeling models. Additionally, our analysis indicates that TCL accelerates training and alleviates the slow training problem associated with complex models. |
| title | An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.13534 |