Responsible AI for Test Equity and Quality: The Duolingo English Test as a Case Study
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914947775070208 |
|---|---|
| author | Burstein, Jill LaFlair, Geoffrey T. Yancey, Kevin von Davier, Alina A. Dotan, Ravit |
| author_facet | Burstein, Jill LaFlair, Geoffrey T. Yancey, Kevin von Davier, Alina A. Dotan, Ravit |
| contents | Artificial intelligence (AI) creates opportunities for assessments, such as efficiencies for item generation and scoring of spoken and written responses. At the same time, it poses risks (such as bias in AI-generated item content). Responsible AI (RAI) practices aim to mitigate risks associated with AI. This chapter addresses the critical role of RAI practices in achieving test quality (appropriateness of test score inferences), and test equity (fairness to all test takers). To illustrate, the chapter presents a case study using the Duolingo English Test (DET), an AI-powered, high-stakes English language assessment. The chapter discusses the DET RAI standards, their development and their relationship to domain-agnostic RAI principles. Further, it provides examples of specific RAI practices, showing how these practices meaningfully address the ethical principles of validity and reliability, fairness, privacy and security, and transparency and accountability standards to ensure test equity and quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07476 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Responsible AI for Test Equity and Quality: The Duolingo English Test as a Case Study Burstein, Jill LaFlair, Geoffrey T. Yancey, Kevin von Davier, Alina A. Dotan, Ravit Computers and Society Artificial Intelligence Computation and Language Artificial intelligence (AI) creates opportunities for assessments, such as efficiencies for item generation and scoring of spoken and written responses. At the same time, it poses risks (such as bias in AI-generated item content). Responsible AI (RAI) practices aim to mitigate risks associated with AI. This chapter addresses the critical role of RAI practices in achieving test quality (appropriateness of test score inferences), and test equity (fairness to all test takers). To illustrate, the chapter presents a case study using the Duolingo English Test (DET), an AI-powered, high-stakes English language assessment. The chapter discusses the DET RAI standards, their development and their relationship to domain-agnostic RAI principles. Further, it provides examples of specific RAI practices, showing how these practices meaningfully address the ethical principles of validity and reliability, fairness, privacy and security, and transparency and accountability standards to ensure test equity and quality. |
| title | Responsible AI for Test Equity and Quality: The Duolingo English Test as a Case Study |
| topic | Computers and Society Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2409.07476 |