Analysis of LLM as a grammatical feature tagger for African American English
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918129354932224 |
|---|---|
| author | Porwal, Rahul Rozet, Alice Houck, Pryce Gowda, Jotsna Moeller, Sarah Tang, Kevin |
| author_facet | Porwal, Rahul Rozet, Alice Houck, Pryce Gowda, Jotsna Moeller, Sarah Tang, Kevin |
| contents | African American English (AAE) presents unique challenges in natural language processing (NLP). This research systematically compares the performance of available NLP models--rule-based, transformer-based, and large language models (LLMs)--capable of identifying key grammatical features of AAE, namely Habitual Be and Multiple Negation. These features were selected for their distinct grammatical complexity and frequency of occurrence. The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. The analysis reveals that while LLMs show promise compared to the baseline, they are influenced by biases such as recency and unrelated features in the text such as formality. This study highlights the necessity for improved model training and architectural adjustments to better accommodate AAE's unique linguistic characteristics. Data and code are available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_06004 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Analysis of LLM as a grammatical feature tagger for African American English Porwal, Rahul Rozet, Alice Houck, Pryce Gowda, Jotsna Moeller, Sarah Tang, Kevin Computation and Language Artificial Intelligence Machine Learning I.2.6; I.2.7; K.4.2; J.4; J.5 African American English (AAE) presents unique challenges in natural language processing (NLP). This research systematically compares the performance of available NLP models--rule-based, transformer-based, and large language models (LLMs)--capable of identifying key grammatical features of AAE, namely Habitual Be and Multiple Negation. These features were selected for their distinct grammatical complexity and frequency of occurrence. The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. The analysis reveals that while LLMs show promise compared to the baseline, they are influenced by biases such as recency and unrelated features in the text such as formality. This study highlights the necessity for improved model training and architectural adjustments to better accommodate AAE's unique linguistic characteristics. Data and code are available. |
| title | Analysis of LLM as a grammatical feature tagger for African American English |
| topic | Computation and Language Artificial Intelligence Machine Learning I.2.6; I.2.7; K.4.2; J.4; J.5 |
| url | https://arxiv.org/abs/2502.06004 |