Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.18893 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910975870894080 |
|---|---|
| author | Schwartz, Reva Chowdhury, Rumman Kundu, Akash Frase, Heather Fadaee, Marzieh David, Tom Waters, Gabriella Taik, Afaf Briggs, Morgan Hall, Patrick Jain, Shomik Yee, Kyra Thomas, Spencer Bhandari, Sundeep Duncan, Paul Thompson, Andrew Carlyle, Maya Lu, Qinghua Holmes, Matthew Skeadas, Theodora |
| author_facet | Schwartz, Reva Chowdhury, Rumman Kundu, Akash Frase, Heather Fadaee, Marzieh David, Tom Waters, Gabriella Taik, Afaf Briggs, Morgan Hall, Patrick Jain, Shomik Yee, Kyra Thomas, Spencer Bhandari, Sundeep Duncan, Paul Thompson, Andrew Carlyle, Maya Lu, Qinghua Holmes, Matthew Skeadas, Theodora |
| contents | Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors. AI capability evaluations can capture detail about first-order effects, such as whether immediate system outputs are accurate, or contain toxic, biased or stereotypical content, but AI's second-order effects, i.e. any long-term outcomes and consequences that may result from AI use in the real world, have become a significant area of interest as the technology becomes embedded in our daily lives. These secondary effects can include shifts in user behavior, societal, cultural and economic ramifications, workforce transformations, and long-term downstream impacts that may result from a broad and growing set of risks. This position paper argues that measuring the indirect and secondary effects of AI will require expansion beyond static, single-turn approaches conducted in silico to include testing paradigms that can capture what actually materializes when people use AI technology in context. Specifically, we describe the need for data and methods that can facilitate contextual awareness and enable downstream interpretation and decision making about AI's secondary effects, and recommend requirements for a new ecosystem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18893 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects Schwartz, Reva Chowdhury, Rumman Kundu, Akash Frase, Heather Fadaee, Marzieh David, Tom Waters, Gabriella Taik, Afaf Briggs, Morgan Hall, Patrick Jain, Shomik Yee, Kyra Thomas, Spencer Bhandari, Sundeep Duncan, Paul Thompson, Andrew Carlyle, Maya Lu, Qinghua Holmes, Matthew Skeadas, Theodora Computers and Society Artificial Intelligence Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors. AI capability evaluations can capture detail about first-order effects, such as whether immediate system outputs are accurate, or contain toxic, biased or stereotypical content, but AI's second-order effects, i.e. any long-term outcomes and consequences that may result from AI use in the real world, have become a significant area of interest as the technology becomes embedded in our daily lives. These secondary effects can include shifts in user behavior, societal, cultural and economic ramifications, workforce transformations, and long-term downstream impacts that may result from a broad and growing set of risks. This position paper argues that measuring the indirect and secondary effects of AI will require expansion beyond static, single-turn approaches conducted in silico to include testing paradigms that can capture what actually materializes when people use AI technology in context. Specifically, we describe the need for data and methods that can facilitate contextual awareness and enable downstream interpretation and decision making about AI's secondary effects, and recommend requirements for a new ecosystem. |
| title | Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects |
| topic | Computers and Society Artificial Intelligence |
| url | https://arxiv.org/abs/2505.18893 |