Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2204.12421 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916745414967296 |
|---|---|
| author | Santiago, Harrison Martin, Joshua Moeller, Sarah Tang, Kevin |
| author_facet | Santiago, Harrison Martin, Joshua Moeller, Sarah Tang, Kevin |
| contents | Recent research has highlighted that natural language processing (NLP) systems exhibit a bias against African American speakers. The bias errors are often caused by poor representation of linguistic features unique to African American English (AAE), due to the relatively low probability of occurrence of many such features in training data. We present a workflow to overcome such bias in the case of habitual "be". Habitual "be" is isomorphic, and therefore ambiguous, with other forms of "be" found in both AAE and other varieties of English. This creates a clear challenge for bias in NLP technologies. To overcome the scarcity, we employ a combination of rule-based filters and data augmentation that generate a corpus balanced between habitual and non-habitual instances. With this balanced corpus, we train unbiased machine learning classifiers, as demonstrated on a corpus of AAE transcribed texts, achieving .65 F$_1$ score disambiguating habitual "be". |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2204_12421 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Disambiguation of morpho-syntactic features of African American English -- the case of habitual be Santiago, Harrison Martin, Joshua Moeller, Sarah Tang, Kevin Computation and Language Recent research has highlighted that natural language processing (NLP) systems exhibit a bias against African American speakers. The bias errors are often caused by poor representation of linguistic features unique to African American English (AAE), due to the relatively low probability of occurrence of many such features in training data. We present a workflow to overcome such bias in the case of habitual "be". Habitual "be" is isomorphic, and therefore ambiguous, with other forms of "be" found in both AAE and other varieties of English. This creates a clear challenge for bias in NLP technologies. To overcome the scarcity, we employ a combination of rule-based filters and data augmentation that generate a corpus balanced between habitual and non-habitual instances. With this balanced corpus, we train unbiased machine learning classifiers, as demonstrated on a corpus of AAE transcribed texts, achieving .65 F$_1$ score disambiguating habitual "be". |
| title | Disambiguation of morpho-syntactic features of African American English -- the case of habitual be |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2204.12421 |