Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.16520 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916883540738048 |
|---|---|
| author | Rizvi, Naba Strickland, Harper Gitelman, Daniel Cooper, Tristan Morales-Flores, Alexis Golden, Michael Kallepalli, Aekta Alurkar, Akshat Owens, Haaset Ahmedi, Saleha Khirwadkar, Isha Munyaka, Imani Ousidhoum, Nedjma |
| author_facet | Rizvi, Naba Strickland, Harper Gitelman, Daniel Cooper, Tristan Morales-Flores, Alexis Golden, Michael Kallepalli, Aekta Alurkar, Akshat Owens, Haaset Ahmedi, Saleha Khirwadkar, Isha Munyaka, Imani Ousidhoum, Nedjma |
| contents | As our understanding of autism and ableism continues to increase, so does our understanding of ableist language towards autistic people. Such language poses a significant challenge in NLP research due to its subtle and context-dependent nature. Yet, detecting anti-autistic ableist language remains underexplored, with existing NLP tools often failing to capture its nuanced expressions. We present AUTALIC, the first benchmark dataset dedicated to the detection of anti-autistic ableist language in context, addressing a significant gap in the field. The dataset comprises 2,400 autism-related sentences collected from Reddit, accompanied by surrounding context, and is annotated by trained experts with backgrounds in neurodiversity. Our comprehensive evaluation reveals that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and align with human judgments, underscoring their limitations in this domain. We publicly release AUTALIC along with the individual annotations which serve as a valuable resource to researchers working on ableism, neurodiversity, and also studying disagreements in annotation tasks. This dataset serves as a crucial step towards developing more inclusive and context-aware NLP systems that better reflect diverse perspectives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_16520 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context Rizvi, Naba Strickland, Harper Gitelman, Daniel Cooper, Tristan Morales-Flores, Alexis Golden, Michael Kallepalli, Aekta Alurkar, Akshat Owens, Haaset Ahmedi, Saleha Khirwadkar, Isha Munyaka, Imani Ousidhoum, Nedjma Computation and Language Artificial Intelligence As our understanding of autism and ableism continues to increase, so does our understanding of ableist language towards autistic people. Such language poses a significant challenge in NLP research due to its subtle and context-dependent nature. Yet, detecting anti-autistic ableist language remains underexplored, with existing NLP tools often failing to capture its nuanced expressions. We present AUTALIC, the first benchmark dataset dedicated to the detection of anti-autistic ableist language in context, addressing a significant gap in the field. The dataset comprises 2,400 autism-related sentences collected from Reddit, accompanied by surrounding context, and is annotated by trained experts with backgrounds in neurodiversity. Our comprehensive evaluation reveals that current language models, including state-of-the-art LLMs, struggle to reliably identify anti-autistic ableism and align with human judgments, underscoring their limitations in this domain. We publicly release AUTALIC along with the individual annotations which serve as a valuable resource to researchers working on ableism, neurodiversity, and also studying disagreements in annotation tasks. This dataset serves as a crucial step towards developing more inclusive and context-aware NLP systems that better reflect diverse perspectives. |
| title | AUTALIC: A Dataset for Anti-AUTistic Ableist Language In Context |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2410.16520 |