The Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classification
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913831307968512 |
|---|---|
| author | Hatefi, Arezoo Vu, Xuan-Son Bhuyan, Monowar Drewes, Frank |
| author_facet | Hatefi, Arezoo Vu, Xuan-Son Bhuyan, Monowar Drewes, Frank |
| contents | We extend and study a semi-supervised model for text classification proposed earlier by Hatefi et al. for classification tasks in which document classes are described by a small number of gold-labeled examples, while the majority of training examples is unlabeled. The model leverages the teacher-student architecture of Meta Pseudo Labels in which a ''teacher'' generates labels for originally unlabeled training data to train the ''student'' and updates its own model iteratively based on the performance of the student on the gold-labeled portion of the data. We extend the original model of Hatefi et al. by an unsupervised pre-training phase based on objective masking, and conduct in-depth performance evaluations of the original model, our extension, and various independent baselines. Experiments are performed using three different datasets in two different languages (English and Swedish). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_06624 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classification Hatefi, Arezoo Vu, Xuan-Son Bhuyan, Monowar Drewes, Frank Computation and Language We extend and study a semi-supervised model for text classification proposed earlier by Hatefi et al. for classification tasks in which document classes are described by a small number of gold-labeled examples, while the majority of training examples is unlabeled. The model leverages the teacher-student architecture of Meta Pseudo Labels in which a ''teacher'' generates labels for originally unlabeled training data to train the ''student'' and updates its own model iteratively based on the performance of the student on the gold-labeled portion of the data. We extend the original model of Hatefi et al. by an unsupervised pre-training phase based on objective masking, and conduct in-depth performance evaluations of the original model, our extension, and various independent baselines. Experiments are performed using three different datasets in two different languages (English and Swedish). |
| title | The Efficiency of Pre-training with Objective Masking in Pseudo Labeling for Semi-Supervised Text Classification |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.06624 |