Don't be a Fool: Pooling Strategies in Offensive Language Detection from User-Intended Adversarial Attacks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916173110575104 |
|---|---|
| author | Yu, Seunguk Choi, Juhwan Kim, Youngbin |
| author_facet | Yu, Seunguk Choi, Juhwan Kim, Youngbin |
| contents | Offensive language detection is an important task for filtering out abusive expressions and improving online user experiences. However, malicious users often attempt to avoid filtering systems through the involvement of textual noises. In this paper, we propose these evasions as user-intended adversarial attacks that insert special symbols or leverage the distinctive features of the Korean language. Furthermore, we introduce simple yet effective pooling strategies in a layer-wise manner to defend against the proposed attacks, focusing on the preceding layers not just the last layer to capture both offensiveness and token embeddings. We demonstrate that these pooling strategies are more robust to performance degradation even when the attack rate is increased, without directly training of such patterns. Notably, we found that models pre-trained on clean texts could achieve a comparable performance in detecting attacked offensive language, to models pre-trained on noisy texts by employing these pooling strategies. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_15467 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Don't be a Fool: Pooling Strategies in Offensive Language Detection from User-Intended Adversarial Attacks Yu, Seunguk Choi, Juhwan Kim, Youngbin Computation and Language Offensive language detection is an important task for filtering out abusive expressions and improving online user experiences. However, malicious users often attempt to avoid filtering systems through the involvement of textual noises. In this paper, we propose these evasions as user-intended adversarial attacks that insert special symbols or leverage the distinctive features of the Korean language. Furthermore, we introduce simple yet effective pooling strategies in a layer-wise manner to defend against the proposed attacks, focusing on the preceding layers not just the last layer to capture both offensiveness and token embeddings. We demonstrate that these pooling strategies are more robust to performance degradation even when the attack rate is increased, without directly training of such patterns. Notably, we found that models pre-trained on clean texts could achieve a comparable performance in detecting attacked offensive language, to models pre-trained on noisy texts by employing these pooling strategies. |
| title | Don't be a Fool: Pooling Strategies in Offensive Language Detection from User-Intended Adversarial Attacks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2403.15467 |