Exploring the Lands Between: A Method for Finding Differences between AI-Decisions and Human Ratings through Generated Samples
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914952784117760 |
|---|---|
| author | Mecke, Lukas Buschek, Daniel Gruenefeld, Uwe Alt, Florian |
| author_facet | Mecke, Lukas Buschek, Daniel Gruenefeld, Uwe Alt, Florian |
| contents | Many important decisions in our everyday lives, such as authentication via biometric models, are made by Artificial Intelligence (AI) systems. These can be in poor alignment with human expectations, and testing them on clear-cut existing data may not be enough to uncover those cases. We propose a method to find samples in the latent space of a generative model, designed to be challenging for a decision-making model with regard to matching human expectations. By presenting those samples to both the decision-making model and human raters, we can identify areas where its decisions align with human intuition and where they contradict it. We apply this method to a face recognition model and collect a dataset of 11,200 human ratings from 100 participants. We discuss findings from our dataset and how our approach can be used to explore the performance of AI models in different contexts and for different user groups. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12801 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring the Lands Between: A Method for Finding Differences between AI-Decisions and Human Ratings through Generated Samples Mecke, Lukas Buschek, Daniel Gruenefeld, Uwe Alt, Florian Human-Computer Interaction Artificial Intelligence Many important decisions in our everyday lives, such as authentication via biometric models, are made by Artificial Intelligence (AI) systems. These can be in poor alignment with human expectations, and testing them on clear-cut existing data may not be enough to uncover those cases. We propose a method to find samples in the latent space of a generative model, designed to be challenging for a decision-making model with regard to matching human expectations. By presenting those samples to both the decision-making model and human raters, we can identify areas where its decisions align with human intuition and where they contradict it. We apply this method to a face recognition model and collect a dataset of 11,200 human ratings from 100 participants. We discuss findings from our dataset and how our approach can be used to explore the performance of AI models in different contexts and for different user groups. |
| title | Exploring the Lands Between: A Method for Finding Differences between AI-Decisions and Human Ratings through Generated Samples |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2409.12801 |