Lon-ea at SemEval-2023 Task 11: A Comparison of Activation Functions for Soft and Hard Label Prediction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910286830632960 |
|---|---|
| author | Hosseini, Peyman Hosseini, Mehran Al-Azzawi, Sana Sabah Liwicki, Marcus Castro, Ignacio Purver, Matthew |
| author_facet | Hosseini, Peyman Hosseini, Mehran Al-Azzawi, Sana Sabah Liwicki, Marcus Castro, Ignacio Purver, Matthew |
| contents | We study the influence of different activation functions in the output layer of deep neural network models for soft and hard label prediction in the learning with disagreement task. In this task, the goal is to quantify the amount of disagreement via predicting soft labels. To predict the soft labels, we use BERT-based preprocessors and encoders and vary the activation function used in the output layer, while keeping other parameters constant. The soft labels are then used for the hard label prediction. The activation functions considered are sigmoid as well as a step-function that is added to the model post-training and a sinusoidal activation function, which is introduced for the first time in this paper. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_02468 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Lon-ea at SemEval-2023 Task 11: A Comparison of Activation Functions for Soft and Hard Label Prediction Hosseini, Peyman Hosseini, Mehran Al-Azzawi, Sana Sabah Liwicki, Marcus Castro, Ignacio Purver, Matthew Computation and Language Machine Learning I.2.7 We study the influence of different activation functions in the output layer of deep neural network models for soft and hard label prediction in the learning with disagreement task. In this task, the goal is to quantify the amount of disagreement via predicting soft labels. To predict the soft labels, we use BERT-based preprocessors and encoders and vary the activation function used in the output layer, while keeping other parameters constant. The soft labels are then used for the hard label prediction. The activation functions considered are sigmoid as well as a step-function that is added to the model post-training and a sinusoidal activation function, which is introduced for the first time in this paper. |
| title | Lon-ea at SemEval-2023 Task 11: A Comparison of Activation Functions for Soft and Hard Label Prediction |
| topic | Computation and Language Machine Learning I.2.7 |
| url | https://arxiv.org/abs/2303.02468 |