What Shapes Participant Data Quality? A Scoping Review and Case Study of Crowdsourced Webcam Eye Tracking in AI Interviews
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910190527315968 |
|---|---|
| author | Lau, Ka Hei Carrie Kasneci, Enkelejda |
| author_facet | Lau, Ka Hei Carrie Kasneci, Enkelejda |
| contents | Webcam-based eye tracking is a cost-effective, scalable method for remote research that effectively reaches broader populations. However, uncontrolled environments and hardware diversity lead to inconsistent data quality in crowdsourcing. To assess current practices, we conducted a scoping review of crowdsourced eye-tracking from 2011-2025. The review confirms fragmented reporting and a lack of established quality benchmarks. To address this lack of predictive insight, we conducted a case study on AI fairness interviews (N=205) using the RealEye platform. Applying Ordered Logistic Regression (OLR) to the platform quality metric, we found that behavioral and technical factors significantly predict data quality. Specifically, within the RealEye platform, higher fixation counts, shorter sessions, and operating system choice yield significantly higher quality grades. Based on this review and platform-specific predictive insights, we provide actionable recommendations to enhance the reliability, transparency, and replicability of future crowdsourced webcam eye tracking in HCI and behavioral science. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_02898 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | What Shapes Participant Data Quality? A Scoping Review and Case Study of Crowdsourced Webcam Eye Tracking in AI Interviews Lau, Ka Hei Carrie Kasneci, Enkelejda Human-Computer Interaction Webcam-based eye tracking is a cost-effective, scalable method for remote research that effectively reaches broader populations. However, uncontrolled environments and hardware diversity lead to inconsistent data quality in crowdsourcing. To assess current practices, we conducted a scoping review of crowdsourced eye-tracking from 2011-2025. The review confirms fragmented reporting and a lack of established quality benchmarks. To address this lack of predictive insight, we conducted a case study on AI fairness interviews (N=205) using the RealEye platform. Applying Ordered Logistic Regression (OLR) to the platform quality metric, we found that behavioral and technical factors significantly predict data quality. Specifically, within the RealEye platform, higher fixation counts, shorter sessions, and operating system choice yield significantly higher quality grades. Based on this review and platform-specific predictive insights, we provide actionable recommendations to enhance the reliability, transparency, and replicability of future crowdsourced webcam eye tracking in HCI and behavioral science. |
| title | What Shapes Participant Data Quality? A Scoping Review and Case Study of Crowdsourced Webcam Eye Tracking in AI Interviews |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2605.02898 |