Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.12148 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913893705580544 |
|---|---|
| author | Di Bonaventura, Chiara McGillivray, Barbara He, Yulan Meroño-Peñuela, Albert |
| author_facet | Di Bonaventura, Chiara McGillivray, Barbara He, Yulan Meroño-Peñuela, Albert |
| contents | Language changes over time, including in the hate speech domain, which evolves quickly following social dynamics and cultural shifts. While NLP research has investigated the impact of language evolution on model training and has proposed several solutions for it, its impact on model benchmarking remains under-explored. Yet, hate speech benchmarks play a crucial role to ensure model safety. In this paper, we empirically evaluate the robustness of 20 language models across two evolving hate speech experiments, and we show the temporal misalignment between static and time-sensitive evaluations. Our findings call for time-sensitive linguistic benchmarks in order to correctly and reliably evaluate language models in the hate speech domain. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12148 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Hatevolution: What Static Benchmarks Don't Tell Us Di Bonaventura, Chiara McGillivray, Barbara He, Yulan Meroño-Peñuela, Albert Computation and Language Language changes over time, including in the hate speech domain, which evolves quickly following social dynamics and cultural shifts. While NLP research has investigated the impact of language evolution on model training and has proposed several solutions for it, its impact on model benchmarking remains under-explored. Yet, hate speech benchmarks play a crucial role to ensure model safety. In this paper, we empirically evaluate the robustness of 20 language models across two evolving hate speech experiments, and we show the temporal misalignment between static and time-sensitive evaluations. Our findings call for time-sensitive linguistic benchmarks in order to correctly and reliably evaluate language models in the hate speech domain. |
| title | Hatevolution: What Static Benchmarks Don't Tell Us |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.12148 |