Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2204.03954 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929680882335744 |
|---|---|
| author | Galke, Lukas Scherp, Ansgar Diera, Andor Karl, Fabian Lin, Bao Xin Khera, Bhakti Meuser, Tim Singhal, Tushar |
| author_facet | Galke, Lukas Scherp, Ansgar Diera, Andor Karl, Fabian Lin, Bao Xin Khera, Bhakti Meuser, Tim Singhal, Tushar |
| contents | We analyze various methods for single-label and multi-label text classification across well-known datasets, categorizing them into bag-of-words, sequence-based, graph-based, and hierarchical approaches. Despite the surge in methods like graph-based models, encoder-only pre-trained language models, notably BERT, remain state-of-the-art. However, recent findings suggest simpler models like logistic regression and trigram-based SVMs outperform newer techniques. While decoder-only generative language models show promise in learning with limited data, they lag behind encoder-only models in performance. We emphasize the superiority of discriminative language models like BERT over generative models for supervised tasks. Additionally, we highlight the literature's lack of robustness in method comparisons, particularly concerning basic hyperparameter optimizations like learning rate in fine-tuning encoder-only language models.
Data availability: The source code is available at https://github.com/drndr/multilabel-text-clf
All datasets used for our experiments are publicly available except the NYT dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2204_03954 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Are We Really Making Much Progress in Text Classification? A Comparative Review Galke, Lukas Scherp, Ansgar Diera, Andor Karl, Fabian Lin, Bao Xin Khera, Bhakti Meuser, Tim Singhal, Tushar Computation and Language We analyze various methods for single-label and multi-label text classification across well-known datasets, categorizing them into bag-of-words, sequence-based, graph-based, and hierarchical approaches. Despite the surge in methods like graph-based models, encoder-only pre-trained language models, notably BERT, remain state-of-the-art. However, recent findings suggest simpler models like logistic regression and trigram-based SVMs outperform newer techniques. While decoder-only generative language models show promise in learning with limited data, they lag behind encoder-only models in performance. We emphasize the superiority of discriminative language models like BERT over generative models for supervised tasks. Additionally, we highlight the literature's lack of robustness in method comparisons, particularly concerning basic hyperparameter optimizations like learning rate in fine-tuning encoder-only language models. Data availability: The source code is available at https://github.com/drndr/multilabel-text-clf All datasets used for our experiments are publicly available except the NYT dataset. |
| title | Are We Really Making Much Progress in Text Classification? A Comparative Review |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2204.03954 |