Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918453555757056 |
|---|---|
| author | So, Yeonkyoung Lee, Gyuseong Jung, Sungmok Lee, Joonhak Kang, JiA Kim, Sangho Lee, Jaejin |
| author_facet | So, Yeonkyoung Lee, Gyuseong Jung, Sungmok Lee, Joonhak Kang, JiA Kim, Sangho Lee, Jaejin |
| contents | Negation is a fundamental linguistic phenomenon that poses ongoing challenges for Large Language Models (LLMs), particularly in tasks requiring deep semantic understanding. Current benchmarks often treat negation as a minor detail within broader tasks, such as natural language inference. Consequently, there is a lack of benchmarks specifically designed to evaluate comprehension of negation. In this work, we introduce Thunder-NUBench, a novel benchmark explicitly created to assess sentence-level understanding of negation in LLMs. Thunder-NUBench goes beyond merely identifying surface-level cues by contrasting standard negation with structurally diverse alternatives, such as local negation, contradiction, and paraphrase. This benchmark includes manually curated sentence-negation pairs and a multiple-choice dataset, allowing for a comprehensive evaluation of models' understanding of negation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_14397 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding So, Yeonkyoung Lee, Gyuseong Jung, Sungmok Lee, Joonhak Kang, JiA Kim, Sangho Lee, Jaejin Computation and Language Negation is a fundamental linguistic phenomenon that poses ongoing challenges for Large Language Models (LLMs), particularly in tasks requiring deep semantic understanding. Current benchmarks often treat negation as a minor detail within broader tasks, such as natural language inference. Consequently, there is a lack of benchmarks specifically designed to evaluate comprehension of negation. In this work, we introduce Thunder-NUBench, a novel benchmark explicitly created to assess sentence-level understanding of negation in LLMs. Thunder-NUBench goes beyond merely identifying surface-level cues by contrasting standard negation with structurally diverse alternatives, such as local negation, contradiction, and paraphrase. This benchmark includes manually curated sentence-negation pairs and a multiple-choice dataset, allowing for a comprehensive evaluation of models' understanding of negation. |
| title | Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.14397 |