Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.24055 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910070015524864 |
|---|---|
| author | Schwartz, Reva Westling, Carina Briggs, Morgan Fadaee, Marzieh Nejadgholi, Isar Holmes, Matthew Rashid, Fariza Carlyle, Maya Taïk, Afaf Wilson, Kyra Douglas, Peter Skeadas, Theodora Waters, Gabriella Chowdhury, Rumman Lacerda, Thiago |
| author_facet | Schwartz, Reva Westling, Carina Briggs, Morgan Fadaee, Marzieh Nejadgholi, Isar Holmes, Matthew Rashid, Fariza Carlyle, Maya Taïk, Afaf Wilson, Kyra Douglas, Peter Skeadas, Theodora Waters, Gabriella Chowdhury, Rumman Lacerda, Thiago |
| contents | This paper proposes CIRCLE, a six-stage, lifecycle-based framework to bridge the reality gap between model-centric performance metrics and AI's materialized outcomes in deployment. Current approaches such as MLOps frameworks and AI model benchmarks offer detailed insights into system stability and model capabilities, but they do not provide decision-makers outside the AI stack with systematic evidence of how these systems actually behave in real-world contexts or affect their organizations over time. CIRCLE operationalizes the Validation phase of TEVV (Test, Evaluation, Verification, and Validation) by formalizing the translation of stakeholder concerns outside the stack into measurable signals. Unlike participatory design, which often remains localized, or algorithmic audits, which are often retrospective, CIRCLE provides a structured, prospective protocol for linking context-sensitive qualitative insights to scalable quantitative metrics. By integrating methods such as field testing, red teaming, and longitudinal studies into a coordinated pipeline, CIRCLE produces systematic knowledge: evidence that is comparable across sites yet sensitive to local context. This, in turn, can enable governance based on materialized downstream effects rather than theoretical capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_24055 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CIRCLE: A Framework for Evaluating AI from a Real-World Lens Schwartz, Reva Westling, Carina Briggs, Morgan Fadaee, Marzieh Nejadgholi, Isar Holmes, Matthew Rashid, Fariza Carlyle, Maya Taïk, Afaf Wilson, Kyra Douglas, Peter Skeadas, Theodora Waters, Gabriella Chowdhury, Rumman Lacerda, Thiago Artificial Intelligence Software Engineering This paper proposes CIRCLE, a six-stage, lifecycle-based framework to bridge the reality gap between model-centric performance metrics and AI's materialized outcomes in deployment. Current approaches such as MLOps frameworks and AI model benchmarks offer detailed insights into system stability and model capabilities, but they do not provide decision-makers outside the AI stack with systematic evidence of how these systems actually behave in real-world contexts or affect their organizations over time. CIRCLE operationalizes the Validation phase of TEVV (Test, Evaluation, Verification, and Validation) by formalizing the translation of stakeholder concerns outside the stack into measurable signals. Unlike participatory design, which often remains localized, or algorithmic audits, which are often retrospective, CIRCLE provides a structured, prospective protocol for linking context-sensitive qualitative insights to scalable quantitative metrics. By integrating methods such as field testing, red teaming, and longitudinal studies into a coordinated pipeline, CIRCLE produces systematic knowledge: evidence that is comparable across sites yet sensitive to local context. This, in turn, can enable governance based on materialized downstream effects rather than theoretical capabilities. |
| title | CIRCLE: A Framework for Evaluating AI from a Real-World Lens |
| topic | Artificial Intelligence Software Engineering |
| url | https://arxiv.org/abs/2602.24055 |