Neural Architecture Search for Sentence Classification with BERT
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929291940331520 |
|---|---|
| author | Kenneweg, Philip Schröder, Sarah Hammer, Barbara |
| author_facet | Kenneweg, Philip Schröder, Sarah Hammer, Barbara |
| contents | Pre training of language models on large text corpora is common practice in Natural Language Processing. Following, fine tuning of these models is performed to achieve the best results on a variety of tasks. In this paper we question the common practice of only adding a single output layer as a classification head on top of the network. We perform an AutoML search to find architectures that outperform the current single layer at only a small compute cost. We validate our classification architecture on a variety of NLP benchmarks from the GLUE dataset. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_18547 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neural Architecture Search for Sentence Classification with BERT Kenneweg, Philip Schröder, Sarah Hammer, Barbara Artificial Intelligence Pre training of language models on large text corpora is common practice in Natural Language Processing. Following, fine tuning of these models is performed to achieve the best results on a variety of tasks. In this paper we question the common practice of only adding a single output layer as a classification head on top of the network. We perform an AutoML search to find architectures that outperform the current single layer at only a small compute cost. We validate our classification architecture on a variety of NLP benchmarks from the GLUE dataset. |
| title | Neural Architecture Search for Sentence Classification with BERT |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2403.18547 |