AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929383003914240 |
|---|---|
| author | Pachinger, Pia Goldzycher, Janis Planitzer, Anna Maria Kusa, Wojciech Hanbury, Allan Neidhardt, Julia |
| author_facet | Pachinger, Pia Goldzycher, Janis Planitzer, Anna Maria Kusa, Wojciech Hanbury, Allan Neidhardt, Julia |
| contents | Model interpretability in toxicity detection greatly profits from token-level annotations. However, currently such annotations are only available in English. We introduce a dataset annotated for offensive language detection sourced from a news forum, notable for its incorporation of the Austrian German dialect, comprising 4,562 user comments. In addition to binary offensiveness classification, we identify spans within each comment constituting vulgar language or representing targets of offensive statements. We evaluate fine-tuned language models as well as large language models in a zero- and few-shot fashion. The results indicate that while fine-tuned models excel in detecting linguistic peculiarities such as vulgar dialect, large language models demonstrate superior performance in detecting offensiveness in AustroTox. We publish the data and code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08080 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection Pachinger, Pia Goldzycher, Janis Planitzer, Anna Maria Kusa, Wojciech Hanbury, Allan Neidhardt, Julia Computation and Language Artificial Intelligence Computers and Society I.2.7 Model interpretability in toxicity detection greatly profits from token-level annotations. However, currently such annotations are only available in English. We introduce a dataset annotated for offensive language detection sourced from a news forum, notable for its incorporation of the Austrian German dialect, comprising 4,562 user comments. In addition to binary offensiveness classification, we identify spans within each comment constituting vulgar language or representing targets of offensive statements. We evaluate fine-tuned language models as well as large language models in a zero- and few-shot fashion. The results indicate that while fine-tuned models excel in detecting linguistic peculiarities such as vulgar dialect, large language models demonstrate superior performance in detecting offensiveness in AustroTox. We publish the data and code. |
| title | AustroTox: A Dataset for Target-Based Austrian German Offensive Language Detection |
| topic | Computation and Language Artificial Intelligence Computers and Society I.2.7 |
| url | https://arxiv.org/abs/2406.08080 |