(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911804390637568 |
|---|---|
| author | Jin, Angela Salehi, Niloufar |
| author_facet | Jin, Angela Salehi, Niloufar |
| contents | Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) -- automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors' non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_13004 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court Jin, Angela Salehi, Niloufar Computers and Society Artificial Intelligence Human-Computer Interaction K.4.0 Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) -- automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors' non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice. |
| title | (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court |
| topic | Computers and Society Artificial Intelligence Human-Computer Interaction K.4.0 |
| url | https://arxiv.org/abs/2403.13004 |